-- Logs begin at Sun 2022-07-17 16:06:35 UTC, end at Sun 2022-07-17 17:02:57 UTC. -- Jul 17 16:38:56.361893 dv-cinder-850015-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 17 16:38:56.371047 dv-cinder-850015-2-iscsi devstack@keystone.service[87183]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 17 16:38:56.373078 dv-cinder-850015-2-iscsi devstack@keystone.service[87183]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Jul 17 16:38:56 2022] *** Jul 17 16:38:56.373078 dv-cinder-850015-2-iscsi devstack@keystone.service[87183]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 17 16:38:56.373078 dv-cinder-850015-2-iscsi devstack@keystone.service[87183]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 17 16:38:56.373078 dv-cinder-850015-2-iscsi devstack@keystone.service[87183]: nodename: dv-cinder-850015-2-iscsi Jul 17 16:38:56.373078 dv-cinder-850015-2-iscsi devstack@keystone.service[87183]: machine: x86_64 Jul 17 16:38:56.373078 dv-cinder-850015-2-iscsi devstack@keystone.service[87183]: clock source: unix Jul 17 16:38:56.373078 dv-cinder-850015-2-iscsi devstack@keystone.service[87183]: pcre jit disabled Jul 17 16:38:56.373078 dv-cinder-850015-2-iscsi devstack@keystone.service[87183]: detected number of CPU cores: 4 Jul 17 16:38:56.373078 dv-cinder-850015-2-iscsi devstack@keystone.service[87183]: current working directory: / Jul 17 16:38:56.373078 dv-cinder-850015-2-iscsi devstack@keystone.service[87183]: detected binary path: /usr/bin/uwsgi-core Jul 17 16:38:56.373078 dv-cinder-850015-2-iscsi devstack@keystone.service[87183]: your processes number limit is 47680 Jul 17 16:38:56.373078 dv-cinder-850015-2-iscsi devstack@keystone.service[87183]: your memory page size is 4096 bytes Jul 17 16:38:56.373078 dv-cinder-850015-2-iscsi devstack@keystone.service[87183]: detected max file descriptor number: 2048 Jul 17 16:38:56.373078 dv-cinder-850015-2-iscsi devstack@keystone.service[87183]: lock engine: pthread robust mutexes Jul 17 16:38:56.374496 dv-cinder-850015-2-iscsi devstack@keystone.service[87183]: thunder lock: enabled Jul 17 16:38:56.374496 dv-cinder-850015-2-iscsi devstack@keystone.service[87183]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 17 16:38:56.374496 dv-cinder-850015-2-iscsi devstack@keystone.service[87183]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 17 16:38:56.418289 dv-cinder-850015-2-iscsi devstack@keystone.service[87183]: Python main interpreter initialized at 0x5578cde609c0 Jul 17 16:38:56.418289 dv-cinder-850015-2-iscsi devstack@keystone.service[87183]: python threads support enabled Jul 17 16:38:56.418289 dv-cinder-850015-2-iscsi devstack@keystone.service[87183]: your server socket listen backlog is limited to 100 connections Jul 17 16:38:56.418289 dv-cinder-850015-2-iscsi devstack@keystone.service[87183]: your mercy for graceful operations on workers is 90 seconds Jul 17 16:38:56.418683 dv-cinder-850015-2-iscsi devstack@keystone.service[87183]: mapped 403077 bytes (393 KB) for 2 cores Jul 17 16:38:56.418760 dv-cinder-850015-2-iscsi devstack@keystone.service[87183]: *** Operational MODE: preforking *** Jul 17 16:38:56.419044 dv-cinder-850015-2-iscsi devstack@keystone.service[87183]: *** uWSGI is running in multiple interpreter mode *** Jul 17 16:38:56.419044 dv-cinder-850015-2-iscsi devstack@keystone.service[87183]: spawned uWSGI master process (pid: 87183) Jul 17 16:38:56.419263 dv-cinder-850015-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 17 16:38:56.420053 dv-cinder-850015-2-iscsi devstack@keystone.service[87183]: spawned uWSGI worker 1 (pid: 87198, cores: 1) Jul 17 16:38:56.421608 dv-cinder-850015-2-iscsi devstack@keystone.service[87183]: spawned uWSGI worker 2 (pid: 87199, cores: 1) Jul 17 16:38:56.421608 dv-cinder-850015-2-iscsi devstack@keystone.service[87183]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 17 16:39:01.978993 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 17 16:39:01.978993 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: from cryptography.utils import int_from_bytes Jul 17 16:39:01.979793 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 17 16:39:01.979793 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: from cryptography.utils import int_from_bytes Jul 17 16:39:01.982161 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 17 16:39:01.982161 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: from cryptography.utils import int_from_bytes Jul 17 16:39:01.982680 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 17 16:39:01.982680 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: from cryptography.utils import int_from_bytes Jul 17 16:39:02.030219 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 17 16:39:02.030426 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 17 16:39:02.030604 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] command line args: [] {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 17 16:39:02.030874 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 17 16:39:02.031053 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ================================================================================ {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 17 16:39:02.031265 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 17 16:39:02.031358 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] admin_token = **** {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.031467 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 17 16:39:02.031553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] config_dir = [] {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.031667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] command line args: [] {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 17 16:39:02.031745 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.031891 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] config_source = [] {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.031989 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 17 16:39:02.032062 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] debug = True {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.032137 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ================================================================================ {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 17 16:39:02.032268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.032515 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] admin_token = **** {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.032589 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] config_dir = [] {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.032652 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.032808 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.032902 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.032975 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] config_source = [] {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.033039 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.033163 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] debug = True {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.033295 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.033405 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.033588 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] list_limit = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.033700 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.033809 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] log_config_append = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.033905 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.034004 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.034105 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.034206 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] log_dir = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.034301 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.034405 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] log_file = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.034512 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] list_limit = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.034612 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.034712 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] log_config_append = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.034815 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.034926 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.034994 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.035108 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] log_dir = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.035221 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.035309 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.035488 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.035628 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] log_file = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.035738 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.035879 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.035988 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.036133 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.036256 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.036382 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.036502 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.036616 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.036738 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.036892 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.037008 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.037128 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.037250 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.037363 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.037477 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.037596 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.037710 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.037840 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.037989 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.038111 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.038233 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.038354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] publish_errors = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.038475 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.038596 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.038709 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.038822 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.038971 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.039147 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.039270 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.039384 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.039505 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.039624 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] standard_threads = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.039744 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.039864 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.039982 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] publish_errors = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.040116 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.040237 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.040351 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.040470 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.040591 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.040710 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] use_journal = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.040854 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.040977 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] use_json = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.040977 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] use_stderr = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.041196 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.041310 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] use_syslog = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.041428 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] standard_threads = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.041550 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.041666 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.041785 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.041898 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.041898 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.042116 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.042306 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] use_journal = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.042428 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.042541 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] use_json = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.042541 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] use_stderr = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.042767 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.042881 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] use_syslog = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.043009 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.043124 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 17 16:39:02.043242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.043357 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.043476 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.043590 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.043715 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.043835 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.043949 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.044062 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.044175 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.044294 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.044418 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.044538 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.044653 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.044766 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.044898 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.045012 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.045137 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.045252 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.045365 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.045484 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.045598 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.045760 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.045883 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.045997 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.046110 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.046223 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.046336 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.046460 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.046576 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.046695 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.046809 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.046922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.047035 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.047160 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.047160 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.047381 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.047500 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.047615 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.047728 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.047922 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.048045 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.048159 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.048159 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.048370 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.048483 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.048604 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.048718 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.048847 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.048964 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.049077 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.049190 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.049315 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.049430 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.049543 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.049663 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.049781 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.049893 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.050019 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.050140 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.050237 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.050331 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.050446 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.050553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.050657 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.050753 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.050846 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.051063 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.051396 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.051528 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.051653 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.051768 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.051882 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.051994 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.052144 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.052267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.052390 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.052506 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.052626 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.052747 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.052898 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.053026 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.053133 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.053246 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.053360 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.053511 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.053638 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.053763 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.053888 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.054023 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.054156 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.054273 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.054388 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.054517 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.054642 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.054765 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.054881 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.055002 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.055123 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.055262 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.055381 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.055503 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.055656 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.055781 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.055924 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.056044 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.056207 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.056338 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.056489 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.056607 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.056722 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.056868 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.057040 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.057170 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.057293 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.057421 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.057578 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.057695 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.057836 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.057967 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.058084 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.058204 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.058379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.058499 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.058614 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.058739 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.058880 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.058999 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.059149 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.059267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.059387 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.059512 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.059649 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.059776 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.059892 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.060006 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.060158 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.060352 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.060471 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.060585 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.060699 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.060892 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.061012 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.061134 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.061251 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.061396 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.061517 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.061657 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.061775 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.061902 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.062031 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.062147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.062261 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.062429 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.062548 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.062663 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.062785 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.062923 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.063041 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.063223 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.063342 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.063462 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.063587 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.063704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.063818 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.063932 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.064046 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.064199 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.064326 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.064448 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.064563 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.064678 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.064830 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.064948 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.065075 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.065190 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.065304 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.065417 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.065531 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.065644 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.065767 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.065914 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.066047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.066754 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.067018 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.067136 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.067284 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.067405 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.067519 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.067639 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.067755 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] auth.external = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.067871 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.067994 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.068110 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.068273 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.068394 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.068509 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.068623 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.068746 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.068914 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] auth.password = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.069042 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.069159 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] auth.token = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.069279 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.069394 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.069517 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.069633 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.069747 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.069862 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.069977 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.070117 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.070242 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.070364 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.070493 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.070610 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.070725 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.070847 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.070963 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.071078 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] credential.caching = True {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.071201 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] auth.external = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.071319 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.071434 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.071548 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.071663 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.071785 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.071908 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.072026 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.072148 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] auth.password = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.072265 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.072388 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] auth.token = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.072505 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.072912 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.073035 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.073151 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.073266 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.073381 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.073496 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.073621 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.073738 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.073853 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.073853 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.074070 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.074205 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.074344 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.074460 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] credential.caching = True {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.074575 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.074691 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.074805 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.074929 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.075047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.075163 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.075279 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] federation.caching = True {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.075394 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.075509 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.075703 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.075822 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.075938 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.076061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.076178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.076294 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.076417 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.076534 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.076650 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.076765 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.076904 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.077020 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.077147 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.077267 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.077383 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.077498 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.077498 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.077714 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.077714 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.077940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.077940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] identity.caching = True {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.078155 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.078155 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] federation.caching = True {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.078369 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.078480 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.078596 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.078711 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.078826 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.078941 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.079056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.079178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.079294 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.079410 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.079525 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.079647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.079764 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.079886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.080003 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.080117 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.080227 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.080346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.080463 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.080463 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.080689 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.080832 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.080951 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.081067 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.081182 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] identity.caching = True {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.081309 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.081434 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.081557 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.081696 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.082431 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.082552 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.082678 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.082678 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.082898 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.083013 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.083128 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.083243 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.083367 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.083484 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.083601 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.083717 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.083832 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.083946 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.084070 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.084187 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.084303 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.084417 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.084532 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.084656 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.084773 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.084925 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.085040 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.085155 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.085270 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.085397 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.085514 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.085631 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.085746 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.085861 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.085989 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.086117 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.086241 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.086357 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.086473 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.086588 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.086703 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.086826 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.086943 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.087058 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.087174 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.087288 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.087439 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.087569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.087686 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.087803 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.087918 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.088031 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.088146 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.088272 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.088389 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.088505 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.088620 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.088734 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.088909 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.089035 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.089153 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.089268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.089383 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.089498 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.089619 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.089744 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.089862 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.089977 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.090093 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.090207 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.090323 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.090450 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.090567 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.090682 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.user = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.090797 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.090914 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.091030 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.091153 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.091276 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.091392 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.091508 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.091623 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.091738 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.091861 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.091979 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.092094 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.092209 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.092324 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.092439 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.092562 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.092679 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.092813 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.092930 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.093047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.093163 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.093291 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.093409 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.093524 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.093640 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.093755 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.093870 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.093993 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.094111 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.user = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.094226 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.094341 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.094457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.094571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.094696 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.094813 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.094929 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.095044 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.095159 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.095275 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.095398 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.095515 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.095630 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.095745 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.095861 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.095976 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.096098 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.096215 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.096331 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.096446 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.096561 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.096676 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.096861 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.096980 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.097095 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.097212 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.097358 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.097467 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.097578 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.097695 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.097812 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.097812 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.098028 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.098152 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.098270 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.098385 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.098500 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.098614 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.098729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.098918 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.099037 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.099152 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.099269 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.099384 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] resource.caching = True {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.099498 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.099622 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.099738 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.099854 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.099969 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.100084 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.100199 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.100327 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.100451 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.100568 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.100683 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.100817 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.100934 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.101061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.101179 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.101179 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.101396 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.101510 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.101625 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.101748 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.101864 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] role.caching = True {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.101979 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.102094 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] role.driver = sql {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.102215 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.102332 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.102456 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.102573 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.102689 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.102832 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.102956 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] resource.caching = True {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.103072 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.103195 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.103313 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.103428 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.103543 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.103659 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.103781 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.103905 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.104022 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.104137 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.104253 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.104367 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.104482 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.104606 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.104723 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.104886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.105004 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.105121 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.105235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.105359 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] role.caching = True {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.105476 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.105592 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] role.driver = sql {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.105592 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.105809 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.105923 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.106047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.106163 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.106278 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.106393 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.106508 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.106624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.106748 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.106864 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.106979 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.107095 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.107209 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.107316 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.107430 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.107546 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.107669 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.107785 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.107900 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.108023 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.108140 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.108256 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.108371 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.108492 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.108609 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.108733 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.108866 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.108866 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.109084 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.109199 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.109314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.109441 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.109558 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.109674 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.109791 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.109905 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.110062 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.110193 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] token.caching = True {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.110314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.110430 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.110544 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.110659 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.110774 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.110899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.111015 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.111131 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.111246 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.111362 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.111476 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.111599 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.111715 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.111832 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.111947 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.112063 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.112178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.112301 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.112417 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.112532 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.112646 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.112761 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.112941 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.113146 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.113272 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.113390 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.113505 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.113620 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.113620 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.113845 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] token.caching = True {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.113963 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.114078 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.114194 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.114309 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.114424 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] database.connection = **** {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.114567 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.114700 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.114817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.114933 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.115051 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.115166 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.115282 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.115405 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.115521 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.115636 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.115751 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.115866 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.115981 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.116105 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.116223 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.116339 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.116455 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.116569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.116685 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.116831 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.116950 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.117049 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.117164 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.117280 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.117395 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.117518 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.117635 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.117750 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.117865 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.117980 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.118095 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] database.connection = **** {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.118224 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.118421 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.118540 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.118656 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.118770 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87198) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 17 16:39:02.122574 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.122848 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.123117 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.123377 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.123635 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.123889 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.124140 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.124397 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.124644 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.124922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.125167 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.125422 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.125711 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.125967 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.126203 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.126450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.126688 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 17 16:39:02.126860 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 17 16:39:02.150780 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87198) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.151009 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87198) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.151234 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87198) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.155475 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.155706 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.155934 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.159875 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87198) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.164920 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.188672 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87198) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.188945 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87198) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.193047 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.193275 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.335346 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87198) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.341357 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.385787 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87198) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.386022 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87198) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.388859 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.389101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.452586 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87198) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.452837 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87198) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.453067 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87198) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.453260 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.453366 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87198) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.453494 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.453574 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87198) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.453708 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87198) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.453787 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.453926 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.454142 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.454357 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.479306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87198) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.479614 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87198) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.479733 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.479821 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87198) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.479897 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.479969 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87198) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.480041 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.480184 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87198) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.480276 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.480445 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87198) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.480539 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.480659 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87198) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.480740 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.480950 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.492662 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87198) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.493483 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.505986 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87198) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.506330 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87198) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.507204 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87198) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.507204 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87198) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.507204 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87198) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.507572 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.507655 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87198) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.507725 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.507807 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87198) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.507880 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.507880 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.507880 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.508067 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.508262 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 17 16:39:02.526199 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.526199 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 17 16:39:02.526199 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.526199 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 17 16:39:02.526199 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.526988 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.527068 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.527140 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 17 16:39:02.527212 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.527283 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.527493 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.527580 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 17 16:39:02.527580 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.527580 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.527766 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.527942 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.538733 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.539014 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.539309 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.539576 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.540276 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.540562 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.540862 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.541144 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.547740 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=87198) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.549642 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=87199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.551218 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=87198) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.551494 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.551778 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.552173 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.552461 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.552729 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.553031 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.553313 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.553391 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=87199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.553587 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.553666 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.553862 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.553943 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.554131 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.554318 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.554626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.554891 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.555163 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.555438 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.555708 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.555977 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.556245 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.581273 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.583471 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87198) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.583748 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.584535 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.586217 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.586448 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.589086 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=87198) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.589330 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.589611 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.591770 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=87199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.592045 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.592325 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.596989 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=87198) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.597291 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.600720 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=87199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.601025 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.602452 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87198) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.602737 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.603018 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.603302 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.603581 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.603856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.604126 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.604395 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.605758 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.606042 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.606323 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.606598 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.606870 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.607142 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.607410 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.607673 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.623419 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.623696 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.627988 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.628201 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87198) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.628319 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.632382 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.632680 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.633857 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.637488 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.637711 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87198) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.637815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.641457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87198) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.642858 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.645987 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.646282 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.646568 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.646851 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.647144 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.647443 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.647779 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.651595 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.651887 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.652168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.652455 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.652747 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.653054 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.666480 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87198) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.666855 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.667129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.667400 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.672406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.672794 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.673074 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.673387 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.675307 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.677073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.677349 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.680494 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=87198) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.680771 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.681085 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.681368 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.681651 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.681794 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.683507 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.683781 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.686884 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=87199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.687156 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.687433 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.687717 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.687998 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.695577 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=87198) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.695949 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.696248 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.696530 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.696829 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.697118 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.697388 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.703756 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=87199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.704029 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.704304 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.704578 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.704881 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.705163 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.705435 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.717626 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=87198) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.720894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87198) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.724364 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.725993 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.726401 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=87199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.727627 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=87198) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.727904 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.728189 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.729736 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.733113 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.734710 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.735593 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.736329 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=87199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.736601 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.736905 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.737345 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=87198) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.740848 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.741139 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.741420 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.741693 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.745051 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.746661 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=87199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.750009 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.750300 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.750578 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.750852 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.750951 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87198) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.751203 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.751484 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.759910 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=87198) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.760193 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.760472 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.760749 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.761057 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.761139 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.761339 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.761435 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.761608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.761687 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.761872 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.762137 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.762404 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.762689 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.762959 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.763229 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.763497 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87198) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.770488 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=87199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 17 16:39:02.770765 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.771042 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.771315 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.771589 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.771875 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.772152 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.772416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.772680 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.772968 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.773255 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.773514 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.773781 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.774045 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 17 16:39:02.802332 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5578cde609c0 pid: 87198 (default app) Jul 17 16:39:02.805734 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b009b-4d12-4738-9094-71c9b31d689c None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:02.806028 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b009b-4d12-4738-9094-71c9b31d689c None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:02.806312 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b009b-4d12-4738-9094-71c9b31d689c None None] PATH_INFO: `/v3` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:02.810554 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 1/1] 10.222.0.40 () {50 vars in 791 bytes} [Sun Jul 17 16:39:02 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:02.812579 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5578cde609c0 pid: 87199 (default app) Jul 17 16:39:11.666267 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f56d5ef-6dc2-4087-b224-5f83e221773e None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:11.666825 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f56d5ef-6dc2-4087-b224-5f83e221773e None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:11.666825 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f56d5ef-6dc2-4087-b224-5f83e221773e None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:11.668286 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 2/2] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:11.676441 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f703816-2a0f-40d1-bd01-d99f70f82c2b None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:11.676864 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f703816-2a0f-40d1-bd01-d99f70f82c2b None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:11.677045 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f703816-2a0f-40d1-bd01-d99f70f82c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:11.746205 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: INFO dbcounter [None req-0f703816-2a0f-40d1-bd01-d99f70f82c2b None None] Registered counter for database keystone Jul 17 16:39:11.767375 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG oslo_db.sqlalchemy.engines [None req-0f703816-2a0f-40d1-bd01-d99f70f82c2b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87199) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 17 16:39:11.773747 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writer thread running {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 17 16:39:11.952195 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG passlib.handlers.bcrypt [None req-0f703816-2a0f-40d1-bd01-d99f70f82c2b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=87199) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 17 16:39:11.952512 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG passlib.handlers.bcrypt [None req-0f703816-2a0f-40d1-bd01-d99f70f82c2b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87199) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 17 16:39:12.006974 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-0f703816-2a0f-40d1-bd01-d99f70f82c2b None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:12.008576 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-0f703816-2a0f-40d1-bd01-d99f70f82c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:12.131402 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 1/3] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:11 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 459 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 17 16:39:12.277571 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f56d5ef-6dc2-4087-b224-5f83e221773e None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:12.349007 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: INFO dbcounter [None req-9f56d5ef-6dc2-4087-b224-5f83e221773e None None] Registered counter for database keystone Jul 17 16:39:12.368435 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG oslo_db.sqlalchemy.engines [None req-9f56d5ef-6dc2-4087-b224-5f83e221773e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87198) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 17 16:39:12.374750 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writer thread running {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 17 16:39:12.494036 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f56d5ef-6dc2-4087-b224-5f83e221773e None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:12.498360 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb07cd-eba2-4960-985a-e5fe08cac78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b2ec585f5d6476389a82d18a223a808', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:12.499013 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb07cd-eba2-4960-985a-e5fe08cac78c admin admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:12.499299 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb07cd-eba2-4960-985a-e5fe08cac78c admin admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:12.499577 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb07cd-eba2-4960-985a-e5fe08cac78c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:12.511279 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb07cd-eba2-4960-985a-e5fe08cac78c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:12.568846 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: WARNING keystone.common.rbac_enforcer.enforcer [None req-76bb07cd-eba2-4960-985a-e5fe08cac78c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 17 16:39:12.569125 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb07cd-eba2-4960-985a-e5fe08cac78c admin admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:12.579530 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 3/4] 10.222.0.40 () {58 vars in 1324 bytes} [Sun Jul 17 16:39:12 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 17 16:39:12.586364 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f703816-2a0f-40d1-bd01-d99f70f82c2b None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:12.600568 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f703816-2a0f-40d1-bd01-d99f70f82c2b None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:12.604943 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2a5662-6c2c-4b71-a152-3a40632f03d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b2ec585f5d6476389a82d18a223a808', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:12.606081 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a5662-6c2c-4b71-a152-3a40632f03d6 admin admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:12.606367 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a5662-6c2c-4b71-a152-3a40632f03d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:12.606645 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a5662-6c2c-4b71-a152-3a40632f03d6 admin admin] PATH_INFO: `/v3/projects/3b2ec585f5d6476389a82d18a223a808` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:12.610683 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2a5662-6c2c-4b71-a152-3a40632f03d6 admin admin] RBAC: Authorizing `identity:get_project(project_id=3b2ec585f5d6476389a82d18a223a808)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:12.663180 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: WARNING keystone.common.rbac_enforcer.enforcer [None req-fd2a5662-6c2c-4b71-a152-3a40632f03d6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 17 16:39:12.663523 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2a5662-6c2c-4b71-a152-3a40632f03d6 admin admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:12.668163 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 2/5] 10.222.0.40 () {56 vars in 1207 bytes} [Sun Jul 17 16:39:12 2022] GET /identity/v3/projects/3b2ec585f5d6476389a82d18a223a808 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:14.672108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd625d8d-56e4-4a7a-81f0-3e36699d450d None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:14.672641 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd625d8d-56e4-4a7a-81f0-3e36699d450d None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:14.672641 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd625d8d-56e4-4a7a-81f0-3e36699d450d None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:14.676893 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 4/6] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:14.682521 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a81bd-8a54-45fb-a024-18b12dde1cb6 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:14.682898 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a81bd-8a54-45fb-a024-18b12dde1cb6 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:14.683150 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559a81bd-8a54-45fb-a024-18b12dde1cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:14.702309 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-559a81bd-8a54-45fb-a024-18b12dde1cb6 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:14.703831 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-559a81bd-8a54-45fb-a024-18b12dde1cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:14.712898 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 3/7] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 17 16:39:14.735184 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd625d8d-56e4-4a7a-81f0-3e36699d450d None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:14.747415 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd625d8d-56e4-4a7a-81f0-3e36699d450d None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:14.751713 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e1ad98-4d37-47fa-bd77-c69762a23a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b2ec585f5d6476389a82d18a223a808', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:14.752353 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1ad98-4d37-47fa-bd77-c69762a23a29 admin admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:14.752644 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1ad98-4d37-47fa-bd77-c69762a23a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:14.753462 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1ad98-4d37-47fa-bd77-c69762a23a29 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:14.761409 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e1ad98-4d37-47fa-bd77-c69762a23a29 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:14.763205 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e1ad98-4d37-47fa-bd77-c69762a23a29 admin admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:14.772913 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 5/8] 10.222.0.40 () {58 vars in 1324 bytes} [Sun Jul 17 16:39:14 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 17 16:39:14.778360 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559a81bd-8a54-45fb-a024-18b12dde1cb6 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:14.787812 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559a81bd-8a54-45fb-a024-18b12dde1cb6 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:14.792169 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1137ba5d-d7e1-42a1-9166-6273c37946f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b2ec585f5d6476389a82d18a223a808', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:14.793058 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137ba5d-d7e1-42a1-9166-6273c37946f6 admin admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:14.793343 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137ba5d-d7e1-42a1-9166-6273c37946f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:14.793621 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137ba5d-d7e1-42a1-9166-6273c37946f6 admin admin] PATH_INFO: `/v3/users/45e7917424634d8897c1eb02702d4bc2` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:14.795008 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1137ba5d-d7e1-42a1-9166-6273c37946f6 admin admin] RBAC: Authorizing `identity:get_user(user_id=45e7917424634d8897c1eb02702d4bc2)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:14.796865 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1137ba5d-d7e1-42a1-9166-6273c37946f6 admin admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:14.798432 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 4/9] 10.222.0.40 () {56 vars in 1198 bytes} [Sun Jul 17 16:39:14 2022] GET /identity/v3/users/45e7917424634d8897c1eb02702d4bc2 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:16.899881 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bef4d-0bd9-43d2-9cc6-a74eab699fff None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:16.900416 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bef4d-0bd9-43d2-9cc6-a74eab699fff None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:16.900510 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bef4d-0bd9-43d2-9cc6-a74eab699fff None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:16.901915 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 6/10] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:39:16.907323 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85a48c-5701-4ea0-93b6-8e5d3e507859 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:16.907745 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85a48c-5701-4ea0-93b6-8e5d3e507859 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:16.907895 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85a48c-5701-4ea0-93b6-8e5d3e507859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:16.923954 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7cdfd0-31a0-4de7-be94-13a5ccc2c066 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:16.924241 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7cdfd0-31a0-4de7-be94-13a5ccc2c066 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:16.924528 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-1e85a48c-5701-4ea0-93b6-8e5d3e507859 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:16.924647 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7cdfd0-31a0-4de7-be94-13a5ccc2c066 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:16.925776 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-1e85a48c-5701-4ea0-93b6-8e5d3e507859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:16.926183 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 7/11] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:39:16.931309 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a282f0-0124-47ea-b39f-0caf20884d28 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:16.931600 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a282f0-0124-47ea-b39f-0caf20884d28 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:16.931885 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a282f0-0124-47ea-b39f-0caf20884d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:16.973219 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG passlib.handlers.bcrypt [None req-d6a282f0-0124-47ea-b39f-0caf20884d28 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=87198) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 17 16:39:16.973680 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG passlib.handlers.bcrypt [None req-d6a282f0-0124-47ea-b39f-0caf20884d28 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87198) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 17 16:39:17.009401 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 5/12] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:39:17.020238 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-d6a282f0-0124-47ea-b39f-0caf20884d28 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:17.022065 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-d6a282f0-0124-47ea-b39f-0caf20884d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:17.032691 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85a48c-5701-4ea0-93b6-8e5d3e507859 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:17.058104 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:17.123606 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 8/13] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 17 16:39:17.140957 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85a48c-5701-4ea0-93b6-8e5d3e507859 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:17.146774 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a282f0-0124-47ea-b39f-0caf20884d28 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:17.176798 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1adcc84-0da6-4e8b-805b-b63e00ff9beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:17.177547 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1adcc84-0da6-4e8b-805b-b63e00ff9beb None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:17.177838 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1adcc84-0da6-4e8b-805b-b63e00ff9beb None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:17.178127 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1adcc84-0da6-4e8b-805b-b63e00ff9beb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:17.190868 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1adcc84-0da6-4e8b-805b-b63e00ff9beb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:39:17.202416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-f1adcc84-0da6-4e8b-805b-b63e00ff9beb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:39:17.202416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:39:17.202416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:39:17.202416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:39:17.202416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:39:17.202416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:39:17.202416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:39:17.202416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:39:17.202416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:39:17.202416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:39:17.202416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:39:17.202416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:39:17.202416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:39:17.202416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:39:17.202416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:39:17.202416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:39:17.203343 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:39:17.203343 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:39:17.203343 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:39:17.203343 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:39:17.203343 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:39:17.203343 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:39:17.203343 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:39:17.203343 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:39:17.203343 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:39:17.203343 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:39:17.203343 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:39:17.203343 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:39:17.203343 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:39:17.203343 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:39:17.203343 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:39:17.203343 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:39:17.204249 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:39:17.204249 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:39:17.204249 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:39:17.204249 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:39:17.204249 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:39:17.204249 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:39:17.204249 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:39:17.204249 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:39:17.204249 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:39:17.204249 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:39:17.204249 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:39:17.204249 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:39:17.204249 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 6/14] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:39:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:39:17.210328 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1adcc84-0da6-4e8b-805b-b63e00ff9beb None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:17.252766 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a282f0-0124-47ea-b39f-0caf20884d28 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:17.255787 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:17.289984 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e550d-49d3-47c2-b526-9bab7b0c46af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:17.290904 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e550d-49d3-47c2-b526-9bab7b0c46af None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:17.291197 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e550d-49d3-47c2-b526-9bab7b0c46af None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:17.291481 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e550d-49d3-47c2-b526-9bab7b0c46af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:17.296960 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:17.316494 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1adcc84-0da6-4e8b-805b-b63e00ff9beb None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:17.351044 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c4ac7-f1da-4aee-bb00-fe20b4934002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:17.351737 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c4ac7-f1da-4aee-bb00-fe20b4934002 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:17.352030 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c4ac7-f1da-4aee-bb00-fe20b4934002 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:17.352315 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c4ac7-f1da-4aee-bb00-fe20b4934002 None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:17.353134 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c4ac7-f1da-4aee-bb00-fe20b4934002 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:17.353456 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c4ac7-f1da-4aee-bb00-fe20b4934002 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:17.354851 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c4ac7-f1da-4aee-bb00-fe20b4934002 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:17.359797 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54e550d-49d3-47c2-b526-9bab7b0c46af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:17.362732 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54e550d-49d3-47c2-b526-9bab7b0c46af None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:17.367632 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 7/15] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:39:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:17.373477 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c4ac7-f1da-4aee-bb00-fe20b4934002 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:17.460923 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 9/16] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:39:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 17 16:39:17.466568 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e550d-49d3-47c2-b526-9bab7b0c46af None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:17.469556 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:17.475726 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c4ac7-f1da-4aee-bb00-fe20b4934002 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:17.510584 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e015f-c50b-4da2-bdb9-c45128a5fd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:17.511452 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e015f-c50b-4da2-bdb9-c45128a5fd91 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:17.511733 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e015f-c50b-4da2-bdb9-c45128a5fd91 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:17.512006 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e015f-c50b-4da2-bdb9-c45128a5fd91 None admin] PATH_INFO: `/v3/users/45e7917424634d8897c1eb02702d4bc2` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:17.513265 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1e015f-c50b-4da2-bdb9-c45128a5fd91 None admin] RBAC: Authorizing `identity:get_user(user_id=45e7917424634d8897c1eb02702d4bc2)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:17.514725 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1e015f-c50b-4da2-bdb9-c45128a5fd91 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:17.516355 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 8/17] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:39:17 2022] GET /identity/v3/users/45e7917424634d8897c1eb02702d4bc2 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:17.522345 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e015f-c50b-4da2-bdb9-c45128a5fd91 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:17.539697 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:17.568655 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e550d-49d3-47c2-b526-9bab7b0c46af None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:17.604280 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb1497-212a-4f11-86be-e366f7a4f376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:17.605147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb1497-212a-4f11-86be-e366f7a4f376 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:17.605437 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb1497-212a-4f11-86be-e366f7a4f376 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:17.605714 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb1497-212a-4f11-86be-e366f7a4f376 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:17.623649 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb1497-212a-4f11-86be-e366f7a4f376 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:17.624319 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e015f-c50b-4da2-bdb9-c45128a5fd91 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:17.625154 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb1497-212a-4f11-86be-e366f7a4f376 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:17.646803 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-72bb1497-212a-4f11-86be-e366f7a4f376 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:39:17.646803 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:39:17.646803 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:39:17.646803 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:39:17.646803 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:39:17.646803 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:39:17.646803 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:39:17.646803 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:39:17.646803 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:39:17.646803 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:39:17.646803 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:39:17.646803 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:39:17.646803 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:39:17.646803 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:39:17.646803 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:39:17.647765 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:39:17.647765 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:39:17.647765 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:39:17.647765 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:39:17.647765 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:39:17.647765 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:39:17.647765 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:39:17.647765 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:39:17.647765 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:39:17.647765 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:39:17.647765 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:39:17.647765 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:39:17.647765 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:39:17.647765 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:39:17.647765 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:39:17.647765 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:39:17.647765 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:39:17.648743 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:39:17.648743 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:39:17.648743 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:39:17.648743 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:39:17.648743 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 10/18] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:39:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:39:17.653738 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb1497-212a-4f11-86be-e366f7a4f376 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:17.660195 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b8d1a-7366-44f8-882c-7820986aec90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:17.661036 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b8d1a-7366-44f8-882c-7820986aec90 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:17.661319 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b8d1a-7366-44f8-882c-7820986aec90 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:17.661606 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b8d1a-7366-44f8-882c-7820986aec90 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:17.662788 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b8d1a-7366-44f8-882c-7820986aec90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:17.664237 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b8d1a-7366-44f8-882c-7820986aec90 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:17.665856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 9/19] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:39:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:17.671523 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b8d1a-7366-44f8-882c-7820986aec90 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:17.711226 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:17.752610 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:17.756046 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb1497-212a-4f11-86be-e366f7a4f376 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:17.772994 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b8d1a-7366-44f8-882c-7820986aec90 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:17.790082 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16a4b73-00df-437c-97f3-73c6e8e3c9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:17.790752 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a4b73-00df-437c-97f3-73c6e8e3c9ce None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:17.791033 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a4b73-00df-437c-97f3-73c6e8e3c9ce None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:17.791312 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16a4b73-00df-437c-97f3-73c6e8e3c9ce None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:17.792037 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a4b73-00df-437c-97f3-73c6e8e3c9ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:17.792349 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a4b73-00df-437c-97f3-73c6e8e3c9ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:17.793822 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16a4b73-00df-437c-97f3-73c6e8e3c9ce None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:17.807190 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad91502-2c1e-4a2e-bf7b-28b643ac6e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:17.807832 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad91502-2c1e-4a2e-bf7b-28b643ac6e0c None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:17.808120 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad91502-2c1e-4a2e-bf7b-28b643ac6e0c None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:17.808400 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad91502-2c1e-4a2e-bf7b-28b643ac6e0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:17.809265 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad91502-2c1e-4a2e-bf7b-28b643ac6e0c None admin] RBAC: Adding query filter params (role.id=1025051975f64f689effa58dc4259e36, scope.domain.id=default, user.id=45e7917424634d8897c1eb02702d4bc2) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:17.809582 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad91502-2c1e-4a2e-bf7b-28b643ac6e0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:17.811039 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad91502-2c1e-4a2e-bf7b-28b643ac6e0c None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:17.811165 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 11/20] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:39:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:17.818938 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 10/21] 10.222.0.40 () {56 vars in 1322 bytes} [Sun Jul 17 16:39:17 2022] GET /identity/v3/role_assignments?user.id=45e7917424634d8897c1eb02702d4bc2&scope.domain.id=default&role.id=1025051975f64f689effa58dc4259e36 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:20.148071 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963dc0da-d7c6-4872-a1b3-f24dab4d231e None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:20.148579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963dc0da-d7c6-4872-a1b3-f24dab4d231e None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:20.148677 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963dc0da-d7c6-4872-a1b3-f24dab4d231e None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:20.150370 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 12/22] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:20.155964 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac6687-74aa-44a1-aa31-d762e6578da3 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:20.156387 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac6687-74aa-44a1-aa31-d762e6578da3 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:20.156575 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac6687-74aa-44a1-aa31-d762e6578da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:20.193925 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-ecac6687-74aa-44a1-aa31-d762e6578da3 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:20.195480 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-ecac6687-74aa-44a1-aa31-d762e6578da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:20.281203 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 11/23] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 17 16:39:20.316541 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963dc0da-d7c6-4872-a1b3-f24dab4d231e None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:20.451231 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963dc0da-d7c6-4872-a1b3-f24dab4d231e None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:20.478240 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:20.498529 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f0b0b-1d2d-4c6d-b1ed-190cfc1c0341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:20.499382 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f0b0b-1d2d-4c6d-b1ed-190cfc1c0341 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:20.499672 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f0b0b-1d2d-4c6d-b1ed-190cfc1c0341 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:20.499950 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f0b0b-1d2d-4c6d-b1ed-190cfc1c0341 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:20.517501 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561f0b0b-1d2d-4c6d-b1ed-190cfc1c0341 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:39:20.527200 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-561f0b0b-1d2d-4c6d-b1ed-190cfc1c0341 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:39:20.527200 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:39:20.527200 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:39:20.527200 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:39:20.527200 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:39:20.527200 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:39:20.527200 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:39:20.527200 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:39:20.527200 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:39:20.527200 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:39:20.527200 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:39:20.527200 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:39:20.527200 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:39:20.527200 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:39:20.527200 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:39:20.527200 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:39:20.528092 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:39:20.528092 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:39:20.528092 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:39:20.528092 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:39:20.528092 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:39:20.528092 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:39:20.528092 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:39:20.528092 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:39:20.528092 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:39:20.528092 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:39:20.528092 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:39:20.528092 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:39:20.528092 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:39:20.528092 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:39:20.528092 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:39:20.528092 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:39:20.528884 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:39:20.528884 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:39:20.528884 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:39:20.528884 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:39:20.528884 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:39:20.528884 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:39:20.528884 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:39:20.528884 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:39:20.528884 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:39:20.528884 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:39:20.528884 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:39:20.528884 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:39:20.528884 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 13/24] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:39:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:39:20.536489 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecac6687-74aa-44a1-aa31-d762e6578da3 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:20.559488 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b338f0d-2821-42c2-bebc-89c5acda6ece None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:20.559790 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b338f0d-2821-42c2-bebc-89c5acda6ece None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:20.560062 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b338f0d-2821-42c2-bebc-89c5acda6ece None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:20.561677 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 14/25] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:39:20.571494 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4662472-267d-4948-beb7-98ba404d25e8 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:20.572395 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4662472-267d-4948-beb7-98ba404d25e8 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:20.572925 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4662472-267d-4948-beb7-98ba404d25e8 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:20.574741 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 15/26] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:39:20.576011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f2a4b-7e45-4ebf-9ce4-6d358b9db33e None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:20.576487 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f2a4b-7e45-4ebf-9ce4-6d358b9db33e None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:20.576991 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f2a4b-7e45-4ebf-9ce4-6d358b9db33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:20.606610 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-c98f2a4b-7e45-4ebf-9ce4-6d358b9db33e None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:20.608142 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-c98f2a4b-7e45-4ebf-9ce4-6d358b9db33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:20.672819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:20.718407 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecac6687-74aa-44a1-aa31-d762e6578da3 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:20.748087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 16/27] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:39:20.749573 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b8593-b5a8-4908-bdab-b0d15c95a3ae None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:20.749875 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b8593-b5a8-4908-bdab-b0d15c95a3ae None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:20.750159 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b8593-b5a8-4908-bdab-b0d15c95a3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:20.770823 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-cb5b8593-b5a8-4908-bdab-b0d15c95a3ae None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:20.771633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74c56fa-cf22-4ac2-9a50-d5b1818d1303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:20.772094 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-cb5b8593-b5a8-4908-bdab-b0d15c95a3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:20.772857 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74c56fa-cf22-4ac2-9a50-d5b1818d1303 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:20.773370 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74c56fa-cf22-4ac2-9a50-d5b1818d1303 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:20.773862 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74c56fa-cf22-4ac2-9a50-d5b1818d1303 None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:20.776042 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74c56fa-cf22-4ac2-9a50-d5b1818d1303 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:20.776566 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74c56fa-cf22-4ac2-9a50-d5b1818d1303 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:20.778229 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74c56fa-cf22-4ac2-9a50-d5b1818d1303 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:20.791363 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 12/28] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:39:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:20.797415 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023caff5-4a8f-4a4c-8e66-c90444d08bff None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:20.797415 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023caff5-4a8f-4a4c-8e66-c90444d08bff None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:20.797415 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023caff5-4a8f-4a4c-8e66-c90444d08bff None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:20.797415 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 13/29] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:20.797415 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023caff5-4a8f-4a4c-8e66-c90444d08bff None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:20.864947 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 17/30] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:39:20.866259 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b8593-b5a8-4908-bdab-b0d15c95a3ae None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:20.917708 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:20.932760 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023caff5-4a8f-4a4c-8e66-c90444d08bff None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:20.982477 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5a92c-a7d6-4c6b-b284-f4300505181b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:20.983108 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5a92c-a7d6-4c6b-b284-f4300505181b None admin] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:20.983398 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5a92c-a7d6-4c6b-b284-f4300505181b None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:20.983680 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5a92c-a7d6-4c6b-b284-f4300505181b None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:20.984729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea5a92c-a7d6-4c6b-b284-f4300505181b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:20.986791 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea5a92c-a7d6-4c6b-b284-f4300505181b None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:20.993533 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:20.993867 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b8593-b5a8-4908-bdab-b0d15c95a3ae None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:21.033396 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd5762-828c-4629-be20-dec6b9d2faf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:21.035941 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd5762-828c-4629-be20-dec6b9d2faf5 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:21.035941 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd5762-828c-4629-be20-dec6b9d2faf5 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:21.035941 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd5762-828c-4629-be20-dec6b9d2faf5 None admin] PATH_INFO: `/v3/users/45e7917424634d8897c1eb02702d4bc2` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:21.037442 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd5762-828c-4629-be20-dec6b9d2faf5 None admin] RBAC: Authorizing `identity:get_user(user_id=45e7917424634d8897c1eb02702d4bc2)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:21.039048 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd5762-828c-4629-be20-dec6b9d2faf5 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:21.041031 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 18/31] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:39:20 2022] GET /identity/v3/users/45e7917424634d8897c1eb02702d4bc2 => generated 247 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:21.045993 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd6148-45bb-4200-a9e8-10e34f2eb3f2 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:21.046292 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd6148-45bb-4200-a9e8-10e34f2eb3f2 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:21.046648 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd6148-45bb-4200-a9e8-10e34f2eb3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:21.065011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-d9cd6148-45bb-4200-a9e8-10e34f2eb3f2 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:21.067301 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-d9cd6148-45bb-4200-a9e8-10e34f2eb3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:21.154868 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 19/32] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:39:21.156522 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd6148-45bb-4200-a9e8-10e34f2eb3f2 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:21.214818 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:21.229765 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 14/33] 10.222.0.40 () {60 vars in 1131 bytes} [Sun Jul 17 16:39:20 2022] POST /identity/v3/roles => generated 219 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:39:21.230986 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5a92c-a7d6-4c6b-b284-f4300505181b None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:21.311456 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd6148-45bb-4200-a9e8-10e34f2eb3f2 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:21.345610 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5a92c-a7d6-4c6b-b284-f4300505181b None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:21.359487 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da7ac1d-d4dd-41fa-b647-0a52a0db8355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:21.360534 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7ac1d-d4dd-41fa-b647-0a52a0db8355 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:21.360917 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7ac1d-d4dd-41fa-b647-0a52a0db8355 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:21.361298 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7ac1d-d4dd-41fa-b647-0a52a0db8355 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:21.374038 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da7ac1d-d4dd-41fa-b647-0a52a0db8355 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:21.375839 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da7ac1d-d4dd-41fa-b647-0a52a0db8355 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:21.393906 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f3df6-0a3a-42b7-94ac-0eeee6001706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:21.395032 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f3df6-0a3a-42b7-94ac-0eeee6001706 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:21.395504 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f3df6-0a3a-42b7-94ac-0eeee6001706 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:21.395955 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f3df6-0a3a-42b7-94ac-0eeee6001706 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:21.402403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-3da7ac1d-d4dd-41fa-b647-0a52a0db8355 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:39:21.402403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:39:21.402403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:39:21.402403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:39:21.402403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:39:21.402403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:39:21.402403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:39:21.402403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:39:21.402403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:39:21.402403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:39:21.402403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:39:21.402403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:39:21.402403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:39:21.402403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:39:21.402403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:39:21.403268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:39:21.403268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:39:21.403268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:39:21.403268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:39:21.403268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:39:21.403268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:39:21.403268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:39:21.403268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:39:21.403268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:39:21.403268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:39:21.403268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:39:21.403268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:39:21.403268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:39:21.403268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:39:21.403268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:39:21.403268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:39:21.403268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:39:21.404054 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:39:21.404054 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:39:21.404054 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:39:21.404054 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:39:21.404054 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 20/34] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:39:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:39:21.404054 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da7ac1d-d4dd-41fa-b647-0a52a0db8355 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:21.406226 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9f3df6-0a3a-42b7-94ac-0eeee6001706 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:21.408087 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9f3df6-0a3a-42b7-94ac-0eeee6001706 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:21.417622 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 15/35] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:39:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:21.418935 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f3df6-0a3a-42b7-94ac-0eeee6001706 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:21.509362 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:21.548451 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:21.556198 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f3df6-0a3a-42b7-94ac-0eeee6001706 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:21.568366 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da7ac1d-d4dd-41fa-b647-0a52a0db8355 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:21.595158 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6f8cb-8ce1-4ed7-9e92-9ddf603b8efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:21.595835 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6f8cb-8ce1-4ed7-9e92-9ddf603b8efe None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:21.596124 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6f8cb-8ce1-4ed7-9e92-9ddf603b8efe None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:21.596971 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6f8cb-8ce1-4ed7-9e92-9ddf603b8efe None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:21.598479 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa6f8cb-8ce1-4ed7-9e92-9ddf603b8efe None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:21.598800 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa6f8cb-8ce1-4ed7-9e92-9ddf603b8efe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:21.600387 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa6f8cb-8ce1-4ed7-9e92-9ddf603b8efe None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:21.610874 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e0083-8eae-4844-aba7-f3c14cc7b98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:21.611565 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e0083-8eae-4844-aba7-f3c14cc7b98f None admin] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:21.611853 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e0083-8eae-4844-aba7-f3c14cc7b98f None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:21.612809 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e0083-8eae-4844-aba7-f3c14cc7b98f None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:21.613735 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e0083-8eae-4844-aba7-f3c14cc7b98f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:21.615145 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3e0083-8eae-4844-aba7-f3c14cc7b98f None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:21.628993 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 16/36] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:39:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:21.629967 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6f8cb-8ce1-4ed7-9e92-9ddf603b8efe None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:21.800817 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 21/37] 10.222.0.40 () {60 vars in 1131 bytes} [Sun Jul 17 16:39:21 2022] POST /identity/v3/roles => generated 225 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:39:21.800817 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e0083-8eae-4844-aba7-f3c14cc7b98f None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:21.892410 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6f8cb-8ce1-4ed7-9e92-9ddf603b8efe None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:21.906335 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3e0083-8eae-4844-aba7-f3c14cc7b98f None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:21.921691 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:21.936715 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590d51ed-c0bb-4ef1-b941-389e56f9375c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:21.937644 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d51ed-c0bb-4ef1-b941-389e56f9375c None admin] REQUEST_METHOD: `PUT` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:21.937939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d51ed-c0bb-4ef1-b941-389e56f9375c None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:21.938214 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590d51ed-c0bb-4ef1-b941-389e56f9375c None admin] PATH_INFO: `/v3/domains/default/users/45e7917424634d8897c1eb02702d4bc2/roles/1025051975f64f689effa58dc4259e36` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:21.940573 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590d51ed-c0bb-4ef1-b941-389e56f9375c None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=45e7917424634d8897c1eb02702d4bc2, role_id=1025051975f64f689effa58dc4259e36)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:21.942109 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590d51ed-c0bb-4ef1-b941-389e56f9375c None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:21.949161 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c7f8d-93a2-4d77-8571-aed77f759c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:21.949855 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c7f8d-93a2-4d77-8571-aed77f759c95 None admin] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:21.950145 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c7f8d-93a2-4d77-8571-aed77f759c95 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:21.950425 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c7f8d-93a2-4d77-8571-aed77f759c95 None admin] PATH_INFO: `/v3/projects` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:21.951153 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c7f8d-93a2-4d77-8571-aed77f759c95 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:21.952751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c7f8d-93a2-4d77-8571-aed77f759c95 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:22.045514 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 17/38] 10.222.0.40 () {58 vars in 1361 bytes} [Sun Jul 17 16:39:21 2022] PUT /identity/v3/domains/default/users/45e7917424634d8897c1eb02702d4bc2/roles/1025051975f64f689effa58dc4259e36 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 17 16:39:22.047127 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd246a93-81fa-4bb9-8e58-2133aac47555 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:22.047426 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd246a93-81fa-4bb9-8e58-2133aac47555 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:22.047710 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd246a93-81fa-4bb9-8e58-2133aac47555 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:22.049013 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 18/39] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:22.051085 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411c793-55c1-4a3c-ba2f-dc071020ef49 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:22.051377 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411c793-55c1-4a3c-ba2f-dc071020ef49 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:22.051701 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411c793-55c1-4a3c-ba2f-dc071020ef49 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:22.053306 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 19/40] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:22.058854 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f8cc2-4257-4d06-a670-ac0082c69007 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:22.059149 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f8cc2-4257-4d06-a670-ac0082c69007 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:22.059434 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359f8cc2-4257-4d06-a670-ac0082c69007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:22.070633 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:22.077952 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-359f8cc2-4257-4d06-a670-ac0082c69007 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:22.091649 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 22/41] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Jul 17 16:39:21 2022] POST /identity/v3/projects => generated 301 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:39:22.091872 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-359f8cc2-4257-4d06-a670-ac0082c69007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:22.091957 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7d407-164c-41e0-80d3-eae7769e8157 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:22.091957 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7d407-164c-41e0-80d3-eae7769e8157 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:22.091957 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7d407-164c-41e0-80d3-eae7769e8157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:22.101632 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-44d7d407-164c-41e0-80d3-eae7769e8157 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:22.103373 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-44d7d407-164c-41e0-80d3-eae7769e8157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:22.194219 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 23/42] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:39:22.205338 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 20/43] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 17 16:39:22.218824 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7d407-164c-41e0-80d3-eae7769e8157 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:22.233203 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f8cc2-4257-4d06-a670-ac0082c69007 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:22.347916 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7d407-164c-41e0-80d3-eae7769e8157 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:22.355161 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359f8cc2-4257-4d06-a670-ac0082c69007 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:22.369531 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:22.382889 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:22.391449 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb7e31e-5d2c-46e9-ba14-87f8ffc19745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:22.392262 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7e31e-5d2c-46e9-ba14-87f8ffc19745 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:22.392371 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5ea38-d9b5-4267-a638-3cdbdb4a1267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:22.392571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7e31e-5d2c-46e9-ba14-87f8ffc19745 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:22.392864 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7e31e-5d2c-46e9-ba14-87f8ffc19745 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:22.393064 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5ea38-d9b5-4267-a638-3cdbdb4a1267 None admin] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:22.393916 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5ea38-d9b5-4267-a638-3cdbdb4a1267 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:22.394212 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5ea38-d9b5-4267-a638-3cdbdb4a1267 None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:22.394927 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb7e31e-5d2c-46e9-ba14-87f8ffc19745 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:22.395131 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d5ea38-d9b5-4267-a638-3cdbdb4a1267 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:22.396415 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb7e31e-5d2c-46e9-ba14-87f8ffc19745 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:22.396564 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d5ea38-d9b5-4267-a638-3cdbdb4a1267 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:22.398295 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 21/44] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:39:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:22.403923 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb7e31e-5d2c-46e9-ba14-87f8ffc19745 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:22.499058 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 24/45] 10.222.0.40 () {60 vars in 1131 bytes} [Sun Jul 17 16:39:22 2022] POST /identity/v3/roles => generated 223 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:39:22.515679 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb7e31e-5d2c-46e9-ba14-87f8ffc19745 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:22.563220 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b009a8-962b-4514-b78e-de1c0df2e0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:22.563220 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b009a8-962b-4514-b78e-de1c0df2e0f9 None admin] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:22.563220 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b009a8-962b-4514-b78e-de1c0df2e0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:22.563220 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b009a8-962b-4514-b78e-de1c0df2e0f9 None admin] PATH_INFO: `/v3/projects` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:22.573534 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b009a8-962b-4514-b78e-de1c0df2e0f9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:22.573534 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b009a8-962b-4514-b78e-de1c0df2e0f9 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:22.654989 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 22/46] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Jul 17 16:39:22 2022] POST /identity/v3/projects => generated 312 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 17 16:39:24.166843 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c44466-7498-4c61-96b5-d53185c34f41 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:24.167396 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c44466-7498-4c61-96b5-d53185c34f41 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:24.167396 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c44466-7498-4c61-96b5-d53185c34f41 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:24.168859 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 25/47] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:24.174222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8717a0-99dc-4635-90b7-86d9f27dca63 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:24.174690 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8717a0-99dc-4635-90b7-86d9f27dca63 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:24.175487 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8717a0-99dc-4635-90b7-86d9f27dca63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:24.192128 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-2b8717a0-99dc-4635-90b7-86d9f27dca63 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:24.193369 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-2b8717a0-99dc-4635-90b7-86d9f27dca63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:24.220741 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:24.261827 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 23/48] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:39:24.285849 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c44466-7498-4c61-96b5-d53185c34f41 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:24.494074 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c44466-7498-4c61-96b5-d53185c34f41 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:24.527667 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdabc583-5adc-444f-a5d2-5a038284e9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:24.528467 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabc583-5adc-444f-a5d2-5a038284e9b6 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:24.528755 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabc583-5adc-444f-a5d2-5a038284e9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:24.529059 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabc583-5adc-444f-a5d2-5a038284e9b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:24.539525 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdabc583-5adc-444f-a5d2-5a038284e9b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:39:24.547285 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-bdabc583-5adc-444f-a5d2-5a038284e9b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:39:24.547285 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:39:24.547285 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:39:24.547285 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:39:24.547285 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:39:24.547285 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:39:24.547285 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:39:24.547285 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:39:24.547285 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:39:24.547285 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:39:24.547285 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:39:24.547285 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:39:24.547285 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:39:24.547285 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:39:24.547285 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:39:24.547285 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:39:24.548180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:39:24.548180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:39:24.548180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:39:24.548180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:39:24.548180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:39:24.548180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:39:24.548180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:39:24.548180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:39:24.548180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:39:24.548180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:39:24.548180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:39:24.548180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:39:24.548180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:39:24.548180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:39:24.548180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:39:24.548180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:39:24.548970 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:39:24.548970 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:39:24.548970 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:39:24.548970 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:39:24.548970 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:39:24.548970 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:39:24.548970 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:39:24.548970 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:39:24.548970 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:39:24.548970 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:39:24.548970 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:39:24.548970 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:39:24.548970 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 26/49] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:39:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:39:24.554961 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8717a0-99dc-4635-90b7-86d9f27dca63 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:24.655761 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8717a0-99dc-4635-90b7-86d9f27dca63 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:24.689937 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b5d6e-1461-43ec-be27-e4ec2334dcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:24.691625 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b5d6e-1461-43ec-be27-e4ec2334dcd2 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:24.691913 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b5d6e-1461-43ec-be27-e4ec2334dcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:24.692198 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b5d6e-1461-43ec-be27-e4ec2334dcd2 None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:24.692722 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df243ec-c729-46cb-a7c1-8a01f0458ba3 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:24.692988 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859b5d6e-1461-43ec-be27-e4ec2334dcd2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:24.693098 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df243ec-c729-46cb-a7c1-8a01f0458ba3 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:24.693304 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859b5d6e-1461-43ec-be27-e4ec2334dcd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:24.693389 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df243ec-c729-46cb-a7c1-8a01f0458ba3 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:24.694700 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 27/50] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:24.694792 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859b5d6e-1461-43ec-be27-e4ec2334dcd2 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:24.699999 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e021f3f-2d49-40e1-9f2b-62b3cd4ded4e None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:24.700296 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e021f3f-2d49-40e1-9f2b-62b3cd4ded4e None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:24.700583 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e021f3f-2d49-40e1-9f2b-62b3cd4ded4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:24.703647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 24/51] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:39:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:24.709865 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b5d6e-1461-43ec-be27-e4ec2334dcd2 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:24.718338 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-5e021f3f-2d49-40e1-9f2b-62b3cd4ded4e None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:24.719602 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-5e021f3f-2d49-40e1-9f2b-62b3cd4ded4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:24.738131 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:24.764638 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:24.788146 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 28/52] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 17 16:39:24.810642 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b5d6e-1461-43ec-be27-e4ec2334dcd2 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:24.811526 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e021f3f-2d49-40e1-9f2b-62b3cd4ded4e None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:24.845950 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9200895f-ebd1-4f82-8242-b6fd9087b1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:24.847111 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200895f-ebd1-4f82-8242-b6fd9087b1d7 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:24.847398 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200895f-ebd1-4f82-8242-b6fd9087b1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:24.847680 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200895f-ebd1-4f82-8242-b6fd9087b1d7 None admin] PATH_INFO: `/v3/users/45e7917424634d8897c1eb02702d4bc2` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:24.848926 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9200895f-ebd1-4f82-8242-b6fd9087b1d7 None admin] RBAC: Authorizing `identity:get_user(user_id=45e7917424634d8897c1eb02702d4bc2)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:24.850383 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9200895f-ebd1-4f82-8242-b6fd9087b1d7 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:24.852105 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 25/53] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:39:24 2022] GET /identity/v3/users/45e7917424634d8897c1eb02702d4bc2 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:24.857573 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9200895f-ebd1-4f82-8242-b6fd9087b1d7 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:24.912510 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e021f3f-2d49-40e1-9f2b-62b3cd4ded4e None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:24.947388 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03fc0f6-603d-4f6b-8336-1ed8bea6b895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:24.948245 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fc0f6-603d-4f6b-8336-1ed8bea6b895 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:24.948541 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fc0f6-603d-4f6b-8336-1ed8bea6b895 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:24.948868 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fc0f6-603d-4f6b-8336-1ed8bea6b895 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:24.950027 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03fc0f6-603d-4f6b-8336-1ed8bea6b895 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:24.951486 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03fc0f6-603d-4f6b-8336-1ed8bea6b895 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:24.953168 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 29/54] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:39:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:24.954092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:24.958808 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03fc0f6-603d-4f6b-8336-1ed8bea6b895 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:24.960720 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9200895f-ebd1-4f82-8242-b6fd9087b1d7 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:24.994645 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f76fbc7-15fc-4e65-b128-44efff1cd7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:24.995479 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76fbc7-15fc-4e65-b128-44efff1cd7c0 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:24.995768 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76fbc7-15fc-4e65-b128-44efff1cd7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:24.996052 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76fbc7-15fc-4e65-b128-44efff1cd7c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:24.997288 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f76fbc7-15fc-4e65-b128-44efff1cd7c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:24.998819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f76fbc7-15fc-4e65-b128-44efff1cd7c0 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:25.006342 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 26/55] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:39:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:25.006496 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:25.007789 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f76fbc7-15fc-4e65-b128-44efff1cd7c0 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:25.058919 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03fc0f6-603d-4f6b-8336-1ed8bea6b895 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:25.093601 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efa763c-849b-4aaa-8934-0ecc3be94aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:25.094378 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa763c-849b-4aaa-8934-0ecc3be94aa5 None admin] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:25.094670 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa763c-849b-4aaa-8934-0ecc3be94aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:25.094951 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa763c-849b-4aaa-8934-0ecc3be94aa5 None admin] PATH_INFO: `/v3/projects` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:25.095670 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efa763c-849b-4aaa-8934-0ecc3be94aa5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:25.097900 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efa763c-849b-4aaa-8934-0ecc3be94aa5 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:25.115254 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f76fbc7-15fc-4e65-b128-44efff1cd7c0 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:25.152991 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851a923-06bd-4369-9f68-66e21aff71cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:25.153657 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851a923-06bd-4369-9f68-66e21aff71cc None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:25.153951 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851a923-06bd-4369-9f68-66e21aff71cc None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:25.154240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851a923-06bd-4369-9f68-66e21aff71cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:25.155077 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9851a923-06bd-4369-9f68-66e21aff71cc None admin] RBAC: Adding query filter params (role.id=1025051975f64f689effa58dc4259e36, scope.domain.id=default, user.id=45e7917424634d8897c1eb02702d4bc2) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:25.155397 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9851a923-06bd-4369-9f68-66e21aff71cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:25.156891 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9851a923-06bd-4369-9f68-66e21aff71cc None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:25.162969 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 27/56] 10.222.0.40 () {56 vars in 1322 bytes} [Sun Jul 17 16:39:25 2022] GET /identity/v3/role_assignments?user.id=45e7917424634d8897c1eb02702d4bc2&scope.domain.id=default&role.id=1025051975f64f689effa58dc4259e36 => generated 530 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:25.184529 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 30/57] 10.222.0.40 () {60 vars in 1140 bytes} [Sun Jul 17 16:39:24 2022] POST /identity/v3/projects => generated 298 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 17 16:39:27.208183 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47bcd6-49cb-438f-84bb-175a45d41838 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:27.208647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47bcd6-49cb-438f-84bb-175a45d41838 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:27.208758 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47bcd6-49cb-438f-84bb-175a45d41838 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:27.210177 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 28/58] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:27.215455 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2d29b-5add-414d-b851-792982492951 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:27.215849 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2d29b-5add-414d-b851-792982492951 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:27.216016 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2d29b-5add-414d-b851-792982492951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:27.232943 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-1fa2d29b-5add-414d-b851-792982492951 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:27.234146 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-1fa2d29b-5add-414d-b851-792982492951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:27.300125 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 31/59] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:39:27.324235 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47bcd6-49cb-438f-84bb-175a45d41838 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:27.352036 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:27.425956 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47bcd6-49cb-438f-84bb-175a45d41838 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:27.458633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def270a-e921-4eca-9db3-d7b3f9e4d08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:27.459364 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def270a-e921-4eca-9db3-d7b3f9e4d08e None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:27.459650 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def270a-e921-4eca-9db3-d7b3f9e4d08e None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:27.459930 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def270a-e921-4eca-9db3-d7b3f9e4d08e None admin] PATH_INFO: `/v3/domains/default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:27.461150 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def270a-e921-4eca-9db3-d7b3f9e4d08e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:27.462593 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def270a-e921-4eca-9db3-d7b3f9e4d08e None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:27.464196 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 29/60] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:39:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:27.469751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2d29b-5add-414d-b851-792982492951 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:27.498348 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:27.569517 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2d29b-5add-414d-b851-792982492951 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:27.603281 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c2995-f3ae-4ce4-8a88-da1b81788d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:27.603955 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c2995-f3ae-4ce4-8a88-da1b81788d3b None admin] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:27.604244 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c2995-f3ae-4ce4-8a88-da1b81788d3b None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:27.604526 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c2995-f3ae-4ce4-8a88-da1b81788d3b None admin] PATH_INFO: `/v3/users` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:27.605279 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c2995-f3ae-4ce4-8a88-da1b81788d3b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:27.606741 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c2995-f3ae-4ce4-8a88-da1b81788d3b None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:27.828342 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 32/61] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 17 16:39:27 2022] POST /identity/v3/users => generated 275 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:39:31.049122 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f17ae-ab65-4b32-9319-3801ded49930 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:31.050232 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f17ae-ab65-4b32-9319-3801ded49930 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:31.050770 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f17ae-ab65-4b32-9319-3801ded49930 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:31.053176 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 30/62] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:31.058729 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e059775-c36d-4f71-b707-71e9837d33e3 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:31.059653 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e059775-c36d-4f71-b707-71e9837d33e3 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:31.060248 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e059775-c36d-4f71-b707-71e9837d33e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:31.083478 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-3e059775-c36d-4f71-b707-71e9837d33e3 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:31.085108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-3e059775-c36d-4f71-b707-71e9837d33e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:31.139019 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd17f5-2f3a-4c5a-b6eb-cb6b3e0102c0 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:31.139648 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd17f5-2f3a-4c5a-b6eb-cb6b3e0102c0 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:31.140147 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd17f5-2f3a-4c5a-b6eb-cb6b3e0102c0 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:31.141771 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 31/63] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:39:31.147511 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a17f7c6-603e-44dc-8fb8-6424620fba09 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:31.148207 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a17f7c6-603e-44dc-8fb8-6424620fba09 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:31.148716 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a17f7c6-603e-44dc-8fb8-6424620fba09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:31.179014 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-7a17f7c6-603e-44dc-8fb8-6424620fba09 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:31.180828 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-7a17f7c6-603e-44dc-8fb8-6424620fba09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:31.182264 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 33/64] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:39:31.218424 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e059775-c36d-4f71-b707-71e9837d33e3 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:31.245176 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:31.327595 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 32/65] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 17 16:39:31.329385 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa6e32-f356-472c-8b68-d6ea10183d02 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:31.329907 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa6e32-f356-472c-8b68-d6ea10183d02 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:31.330401 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa6e32-f356-472c-8b68-d6ea10183d02 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:31.331901 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 33/66] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:31.340126 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1dfee-7fd0-4a8a-b8f6-7d2f3c8dcb42 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:31.340745 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1dfee-7fd0-4a8a-b8f6-7d2f3c8dcb42 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:31.341279 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1dfee-7fd0-4a8a-b8f6-7d2f3c8dcb42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:31.368290 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-5cf1dfee-7fd0-4a8a-b8f6-7d2f3c8dcb42 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:31.370126 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-5cf1dfee-7fd0-4a8a-b8f6-7d2f3c8dcb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:31.372294 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e059775-c36d-4f71-b707-71e9837d33e3 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:31.389410 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:31.416482 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ca03a-75a2-42bf-8a9a-b6ec9db50a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:31.417956 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ca03a-75a2-42bf-8a9a-b6ec9db50a18 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:31.418393 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ca03a-75a2-42bf-8a9a-b6ec9db50a18 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:31.419155 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ca03a-75a2-42bf-8a9a-b6ec9db50a18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:31.433080 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ca03a-75a2-42bf-8a9a-b6ec9db50a18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:39:31.443896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-f78ca03a-75a2-42bf-8a9a-b6ec9db50a18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:39:31.443896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:39:31.443896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:39:31.443896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:39:31.443896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:39:31.443896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:39:31.443896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:39:31.443896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:39:31.443896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:39:31.443896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:39:31.443896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:39:31.443896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:39:31.443896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:39:31.443896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:39:31.443896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:39:31.443896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:39:31.444997 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:39:31.444997 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:39:31.444997 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:39:31.444997 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:39:31.444997 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:39:31.444997 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:39:31.444997 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:39:31.444997 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:39:31.444997 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:39:31.444997 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:39:31.444997 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:39:31.444997 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:39:31.444997 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:39:31.444997 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:39:31.444997 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:39:31.444997 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:39:31.445907 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:39:31.445907 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:39:31.445907 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:39:31.445907 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:39:31.445907 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:39:31.445907 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:39:31.445907 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:39:31.445907 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:39:31.445907 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:39:31.445907 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:39:31.445907 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:39:31.445907 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:39:31.446489 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 34/67] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:39:31.447478 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf1dfee-7fd0-4a8a-b8f6-7d2f3c8dcb42 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:31.455281 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 34/68] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:39:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:39:31.461731 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ca03a-75a2-42bf-8a9a-b6ec9db50a18 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:31.545820 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:31.574764 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ca03a-75a2-42bf-8a9a-b6ec9db50a18 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:31.575888 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf1dfee-7fd0-4a8a-b8f6-7d2f3c8dcb42 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:31.613872 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59020a89-b45a-4b61-a66e-4c13dccaea24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:31.614476 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59020a89-b45a-4b61-a66e-4c13dccaea24 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:31.614769 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59020a89-b45a-4b61-a66e-4c13dccaea24 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:31.615052 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59020a89-b45a-4b61-a66e-4c13dccaea24 None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:31.615815 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59020a89-b45a-4b61-a66e-4c13dccaea24 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:31.616191 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59020a89-b45a-4b61-a66e-4c13dccaea24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:31.617342 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a42fd-8242-4e4d-abba-08d327a3c774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:31.617639 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59020a89-b45a-4b61-a66e-4c13dccaea24 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:31.618151 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a42fd-8242-4e4d-abba-08d327a3c774 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:31.618440 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a42fd-8242-4e4d-abba-08d327a3c774 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:31.618719 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a42fd-8242-4e4d-abba-08d327a3c774 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:31.633779 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 35/69] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:39:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:31.633779 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b5d42-354f-476e-b332-5d7ba16b0199 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:31.633779 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b5d42-354f-476e-b332-5d7ba16b0199 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:31.633779 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b5d42-354f-476e-b332-5d7ba16b0199 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:31.633779 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 36/70] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:31.633779 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b5d42-354f-476e-b332-5d7ba16b0199 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:31.635632 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a42fd-8242-4e4d-abba-08d327a3c774 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:39:31.643687 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-746a42fd-8242-4e4d-abba-08d327a3c774 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:39:31.643687 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:39:31.643687 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:39:31.643687 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:39:31.643687 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:39:31.643687 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:39:31.643687 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:39:31.643687 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:39:31.643687 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:39:31.643687 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:39:31.643687 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:39:31.643687 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:39:31.643687 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:39:31.643687 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:39:31.643687 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:39:31.643687 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:39:31.644550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:39:31.644550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:39:31.644550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:39:31.644550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:39:31.644550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:39:31.644550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:39:31.644550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:39:31.644550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:39:31.644550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:39:31.644550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:39:31.644550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:39:31.644550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:39:31.644550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:39:31.644550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:39:31.644550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:39:31.644550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:39:31.645868 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:39:31.645868 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:39:31.645868 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:39:31.645868 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:39:31.645868 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:39:31.645868 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:39:31.645868 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:39:31.645868 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:39:31.645868 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:39:31.645868 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:39:31.645868 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:39:31.645868 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:39:31.645868 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 35/71] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:39:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:39:31.646747 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13679697-a390-46ae-8329-185b6a43cccd None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:31.647105 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13679697-a390-46ae-8329-185b6a43cccd None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:31.647394 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13679697-a390-46ae-8329-185b6a43cccd None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:31.648846 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 36/72] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:31.649402 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13679697-a390-46ae-8329-185b6a43cccd None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:31.666187 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:31.745013 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b5d42-354f-476e-b332-5d7ba16b0199 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:31.755041 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13679697-a390-46ae-8329-185b6a43cccd None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:31.780420 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5709d64-01d3-4ffd-9bcb-f087b1cb6235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:31.781278 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5709d64-01d3-4ffd-9bcb-f087b1cb6235 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:31.781571 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5709d64-01d3-4ffd-9bcb-f087b1cb6235 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:31.781863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5709d64-01d3-4ffd-9bcb-f087b1cb6235 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:31.784655 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:31.791246 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a0f69b-4777-4fd8-84c4-981c1be306bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:31.797366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0f69b-4777-4fd8-84c4-981c1be306bf None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:31.797366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0f69b-4777-4fd8-84c4-981c1be306bf None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:31.797366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0f69b-4777-4fd8-84c4-981c1be306bf None admin] PATH_INFO: `/v3/users/45e7917424634d8897c1eb02702d4bc2` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:31.797676 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5709d64-01d3-4ffd-9bcb-f087b1cb6235 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:39:31.797778 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a0f69b-4777-4fd8-84c4-981c1be306bf None admin] RBAC: Authorizing `identity:get_user(user_id=45e7917424634d8897c1eb02702d4bc2)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:31.797778 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a0f69b-4777-4fd8-84c4-981c1be306bf None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:31.797778 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 37/73] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:39:31 2022] GET /identity/v3/users/45e7917424634d8897c1eb02702d4bc2 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:31.798459 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e80e1c7-3814-4b42-954c-1eb442161d3e None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:31.798754 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e80e1c7-3814-4b42-954c-1eb442161d3e None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:31.799037 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e80e1c7-3814-4b42-954c-1eb442161d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:31.801860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-f5709d64-01d3-4ffd-9bcb-f087b1cb6235 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 17 16:39:31.801860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:39:31.801860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:39:31.801860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:39:31.801860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:39:31.801860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:39:31.801860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:39:31.801860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:39:31.801860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:39:31.801860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:39:31.801860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:39:31.801860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:39:31.801860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:39:31.801860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:39:31.801860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:39:31.801860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:39:31.802690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:39:31.802690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:39:31.802690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:39:31.802690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:39:31.802690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:39:31.802690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:39:31.802690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:39:31.802690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:39:31.802690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:39:31.802690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:39:31.802690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:39:31.802690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:39:31.802690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:39:31.802690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:39:31.802690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:39:31.802690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:39:31.803481 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:39:31.803481 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:39:31.803481 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:39:31.803481 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:39:31.803481 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:39:31.803481 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:39:31.803481 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:39:31.803481 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:39:31.803481 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:39:31.803481 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:39:31.803481 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 17 16:39:31.803481 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:39:31.803481 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 37/74] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 17 16:39:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:39:31.808331 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5709d64-01d3-4ffd-9bcb-f087b1cb6235 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:31.817373 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-0e80e1c7-3814-4b42-954c-1eb442161d3e None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:31.818801 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-0e80e1c7-3814-4b42-954c-1eb442161d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:31.896842 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 38/75] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:39:31.896842 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54886571-c247-4276-b1b8-f44898658597 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:31.896842 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54886571-c247-4276-b1b8-f44898658597 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:31.896842 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54886571-c247-4276-b1b8-f44898658597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:31.911475 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-54886571-c247-4276-b1b8-f44898658597 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:31.912638 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5709d64-01d3-4ffd-9bcb-f087b1cb6235 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:31.912873 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-54886571-c247-4276-b1b8-f44898658597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:31.921035 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:31.948534 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d2734-449a-4a63-9d75-1a64d107bbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:31.949259 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d2734-449a-4a63-9d75-1a64d107bbec None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:31.949550 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d2734-449a-4a63-9d75-1a64d107bbec None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:31.949833 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d2734-449a-4a63-9d75-1a64d107bbec None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:31.950601 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9d2734-449a-4a63-9d75-1a64d107bbec None admin] RBAC: Adding query filter params (name=member) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:31.950916 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9d2734-449a-4a63-9d75-1a64d107bbec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:31.952327 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9d2734-449a-4a63-9d75-1a64d107bbec None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:31.962283 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 38/76] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 17 16:39:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:31.963045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d2734-449a-4a63-9d75-1a64d107bbec None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:31.987198 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 39/77] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:39:31.988092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54886571-c247-4276-b1b8-f44898658597 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:32.036717 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:32.079925 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d2734-449a-4a63-9d75-1a64d107bbec None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:32.091221 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54886571-c247-4276-b1b8-f44898658597 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:32.115987 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4c5a3-6f64-4ea2-b6bb-d8c99931ec24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:32.116647 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4c5a3-6f64-4ea2-b6bb-d8c99931ec24 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:32.116978 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4c5a3-6f64-4ea2-b6bb-d8c99931ec24 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:32.117258 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4c5a3-6f64-4ea2-b6bb-d8c99931ec24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:32.125473 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac22396-719a-4d3d-9491-0fec76920ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:32.126155 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac22396-719a-4d3d-9491-0fec76920ee9 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:32.126443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac22396-719a-4d3d-9491-0fec76920ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:32.126725 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac22396-719a-4d3d-9491-0fec76920ee9 None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:32.127493 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac22396-719a-4d3d-9491-0fec76920ee9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:32.127806 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac22396-719a-4d3d-9491-0fec76920ee9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:32.129920 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac22396-719a-4d3d-9491-0fec76920ee9 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:32.138130 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 40/78] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:39:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:32.138943 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac22396-719a-4d3d-9491-0fec76920ee9 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:32.143156 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:32.197457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4c5a3-6f64-4ea2-b6bb-d8c99931ec24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:32.198963 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4c5a3-6f64-4ea2-b6bb-d8c99931ec24 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:32.257815 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:32.259228 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac22396-719a-4d3d-9491-0fec76920ee9 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:32.294246 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa8d3c8-b69a-4a7d-858f-e97ecc0dfe55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:32.295056 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8d3c8-b69a-4a7d-858f-e97ecc0dfe55 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:32.295348 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8d3c8-b69a-4a7d-858f-e97ecc0dfe55 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:32.295636 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8d3c8-b69a-4a7d-858f-e97ecc0dfe55 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:32.296859 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa8d3c8-b69a-4a7d-858f-e97ecc0dfe55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:32.298399 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa8d3c8-b69a-4a7d-858f-e97ecc0dfe55 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:32.300056 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 41/79] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:39:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:32.301429 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa8d3c8-b69a-4a7d-858f-e97ecc0dfe55 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:32.302297 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 39/80] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:39:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 17 16:39:32.303152 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4c5a3-6f64-4ea2-b6bb-d8c99931ec24 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:32.378650 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:32.403739 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4c5a3-6f64-4ea2-b6bb-d8c99931ec24 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:32.409213 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa8d3c8-b69a-4a7d-858f-e97ecc0dfe55 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:32.438373 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd0562-b93b-48f7-8111-98a5f85812a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:32.439102 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd0562-b93b-48f7-8111-98a5f85812a3 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:32.439410 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd0562-b93b-48f7-8111-98a5f85812a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:32.439687 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd0562-b93b-48f7-8111-98a5f85812a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:32.447148 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61beb1-c568-4345-8de5-58c0a3d7c127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:32.448021 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61beb1-c568-4345-8de5-58c0a3d7c127 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:32.448312 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61beb1-c568-4345-8de5-58c0a3d7c127 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:32.448592 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61beb1-c568-4345-8de5-58c0a3d7c127 None admin] PATH_INFO: `/v3/users/f6118de0629f4d08aa5388377f28b7ac` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:32.451294 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd0562-b93b-48f7-8111-98a5f85812a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:39:32.459377 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-d4bd0562-b93b-48f7-8111-98a5f85812a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:39:32.459377 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:39:32.459377 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:39:32.459377 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:39:32.459377 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:39:32.459377 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:39:32.459377 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:39:32.459377 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:39:32.459377 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:39:32.459377 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:39:32.459377 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:39:32.459377 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:39:32.459377 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:39:32.459377 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:39:32.459377 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:39:32.459377 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:39:32.460268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:39:32.460268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:39:32.460268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:39:32.460268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:39:32.460268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:39:32.460268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:39:32.460268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:39:32.460268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:39:32.460268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:39:32.460268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:39:32.460268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:39:32.460268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:39:32.460268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:39:32.460268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:39:32.460268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:39:32.460268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:39:32.461473 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:39:32.461473 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:39:32.461473 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:39:32.461473 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:39:32.461473 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:39:32.461473 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:39:32.461473 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:39:32.461473 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:39:32.461473 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:39:32.461473 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:39:32.461473 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:39:32.461473 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:39:32.461473 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 40/81] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:39:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:39:32.462137 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd0562-b93b-48f7-8111-98a5f85812a3 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:32.463980 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba61beb1-c568-4345-8de5-58c0a3d7c127 None admin] RBAC: Authorizing `identity:get_user(user_id=f6118de0629f4d08aa5388377f28b7ac)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:32.465661 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba61beb1-c568-4345-8de5-58c0a3d7c127 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:32.467439 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 42/82] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:39:32 2022] GET /identity/v3/users/f6118de0629f4d08aa5388377f28b7ac => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:32.468200 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61beb1-c568-4345-8de5-58c0a3d7c127 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:32.481187 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:32.563652 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd0562-b93b-48f7-8111-98a5f85812a3 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:32.574725 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61beb1-c568-4345-8de5-58c0a3d7c127 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:32.599517 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d991dfc2-393d-4b46-ad2e-39874559e12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:32.600384 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991dfc2-393d-4b46-ad2e-39874559e12e None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:32.600668 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991dfc2-393d-4b46-ad2e-39874559e12e None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:32.600980 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991dfc2-393d-4b46-ad2e-39874559e12e None admin] PATH_INFO: `/v3/users/f6118de0629f4d08aa5388377f28b7ac` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:32.602173 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d991dfc2-393d-4b46-ad2e-39874559e12e None admin] RBAC: Authorizing `identity:get_user(user_id=f6118de0629f4d08aa5388377f28b7ac)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:32.603630 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d991dfc2-393d-4b46-ad2e-39874559e12e None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:32.604904 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:32.605313 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 41/83] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:39:32 2022] GET /identity/v3/users/f6118de0629f4d08aa5388377f28b7ac => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:32.606582 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d991dfc2-393d-4b46-ad2e-39874559e12e None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:32.611149 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c57fe0-b192-460b-bb29-951b86420f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:32.611907 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c57fe0-b192-460b-bb29-951b86420f60 None admin] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:32.612224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c57fe0-b192-460b-bb29-951b86420f60 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:32.612503 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c57fe0-b192-460b-bb29-951b86420f60 None admin] PATH_INFO: `/v3/users` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:32.613271 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c57fe0-b192-460b-bb29-951b86420f60 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:32.615272 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c57fe0-b192-460b-bb29-951b86420f60 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:32.691836 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:32.706486 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d991dfc2-393d-4b46-ad2e-39874559e12e None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:32.739549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f28cd-2dec-46e9-a485-32aed2f7e3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:32.740422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f28cd-2dec-46e9-a485-32aed2f7e3c2 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:32.740710 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f28cd-2dec-46e9-a485-32aed2f7e3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:32.741017 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f28cd-2dec-46e9-a485-32aed2f7e3c2 None admin] PATH_INFO: `/v3/projects/fde6b97a4d09416a81257067f1d1b919` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:32.742233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f28cd-2dec-46e9-a485-32aed2f7e3c2 None admin] RBAC: Authorizing `identity:get_project(project_id=fde6b97a4d09416a81257067f1d1b919)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:32.743789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f28cd-2dec-46e9-a485-32aed2f7e3c2 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:32.744518 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 43/84] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 17 16:39:32 2022] POST /identity/v3/users => generated 289 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:39:32.749759 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c57fe0-b192-460b-bb29-951b86420f60 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:32.750116 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 42/85] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:39:32 2022] GET /identity/v3/projects/fde6b97a4d09416a81257067f1d1b919 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:32.750116 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f28cd-2dec-46e9-a485-32aed2f7e3c2 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:32.876583 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c57fe0-b192-460b-bb29-951b86420f60 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:32.878261 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f28cd-2dec-46e9-a485-32aed2f7e3c2 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:32.921735 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11bbf9c-6545-4045-aa90-fd07dc1b2a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:32.922867 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11bbf9c-6545-4045-aa90-fd07dc1b2a27 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:32.922867 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11bbf9c-6545-4045-aa90-fd07dc1b2a27 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:32.923138 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11bbf9c-6545-4045-aa90-fd07dc1b2a27 None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:32.924594 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11bbf9c-6545-4045-aa90-fd07dc1b2a27 None admin] RBAC: Adding query filter params (name=member) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:32.924594 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11bbf9c-6545-4045-aa90-fd07dc1b2a27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:32.924945 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebffc53-55e6-4a64-92f4-1a2ceea3e282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:32.925600 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11bbf9c-6545-4045-aa90-fd07dc1b2a27 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:32.925693 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebffc53-55e6-4a64-92f4-1a2ceea3e282 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:32.928223 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebffc53-55e6-4a64-92f4-1a2ceea3e282 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:32.928223 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebffc53-55e6-4a64-92f4-1a2ceea3e282 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:32.935407 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 44/86] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 17 16:39:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:32.936262 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11bbf9c-6545-4045-aa90-fd07dc1b2a27 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:32.959822 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:33.019212 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebffc53-55e6-4a64-92f4-1a2ceea3e282 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:33.019923 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:33.021554 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebffc53-55e6-4a64-92f4-1a2ceea3e282 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:33.086491 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11bbf9c-6545-4045-aa90-fd07dc1b2a27 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:33.136270 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365cbfd9-4dae-44d4-819a-b1c8436fada1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:33.137095 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365cbfd9-4dae-44d4-819a-b1c8436fada1 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:33.137277 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365cbfd9-4dae-44d4-819a-b1c8436fada1 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:33.137574 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365cbfd9-4dae-44d4-819a-b1c8436fada1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:33.180288 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 43/87] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:39:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 17 16:39:33.181224 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebffc53-55e6-4a64-92f4-1a2ceea3e282 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:33.243468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365cbfd9-4dae-44d4-819a-b1c8436fada1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:33.245420 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365cbfd9-4dae-44d4-819a-b1c8436fada1 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:33.263427 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:33.312529 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:33.326463 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebffc53-55e6-4a64-92f4-1a2ceea3e282 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:33.371095 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a76f0-50a6-41b8-b129-1bcfa9055e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:33.372257 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a76f0-50a6-41b8-b129-1bcfa9055e9e None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:33.372739 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a76f0-50a6-41b8-b129-1bcfa9055e9e None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:33.373250 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a76f0-50a6-41b8-b129-1bcfa9055e9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:33.374358 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495a76f0-50a6-41b8-b129-1bcfa9055e9e None admin] RBAC: Adding query filter params (role.id=1025051975f64f689effa58dc4259e36, scope.project.id=fde6b97a4d09416a81257067f1d1b919, user.id=45e7917424634d8897c1eb02702d4bc2) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:33.374893 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495a76f0-50a6-41b8-b129-1bcfa9055e9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:33.376575 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495a76f0-50a6-41b8-b129-1bcfa9055e9e None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:33.385360 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 45/88] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:39:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 17 16:39:33.386506 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365cbfd9-4dae-44d4-819a-b1c8436fada1 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:33.386905 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 44/89] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:39:33 2022] GET /identity/v3/role_assignments?user.id=45e7917424634d8897c1eb02702d4bc2&scope.project.id=fde6b97a4d09416a81257067f1d1b919&role.id=1025051975f64f689effa58dc4259e36 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:33.388076 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a76f0-50a6-41b8-b129-1bcfa9055e9e None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:33.503265 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495a76f0-50a6-41b8-b129-1bcfa9055e9e None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:33.507566 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365cbfd9-4dae-44d4-819a-b1c8436fada1 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:33.524237 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:33.542524 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a2264-cbaa-4d8d-80b1-3fc7649a3c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:33.543816 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a2264-cbaa-4d8d-80b1-3fc7649a3c6b None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:33.544386 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a2264-cbaa-4d8d-80b1-3fc7649a3c6b None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:33.544852 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a2264-cbaa-4d8d-80b1-3fc7649a3c6b None admin] PATH_INFO: `/v3/projects/fde6b97a4d09416a81257067f1d1b919` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:33.549179 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2a2264-cbaa-4d8d-80b1-3fc7649a3c6b None admin] RBAC: Authorizing `identity:get_project(project_id=fde6b97a4d09416a81257067f1d1b919)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:33.549310 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129ffcf-ec1f-4697-b26f-1eba210037aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:33.549414 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2a2264-cbaa-4d8d-80b1-3fc7649a3c6b None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:33.549483 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129ffcf-ec1f-4697-b26f-1eba210037aa None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:33.549782 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129ffcf-ec1f-4697-b26f-1eba210037aa None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:33.550069 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129ffcf-ec1f-4697-b26f-1eba210037aa None admin] PATH_INFO: `/v3/users/f6118de0629f4d08aa5388377f28b7ac` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:33.551976 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f129ffcf-ec1f-4697-b26f-1eba210037aa None admin] RBAC: Authorizing `identity:get_user(user_id=f6118de0629f4d08aa5388377f28b7ac)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:33.552817 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 45/90] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:39:33 2022] GET /identity/v3/projects/fde6b97a4d09416a81257067f1d1b919 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:33.553506 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f129ffcf-ec1f-4697-b26f-1eba210037aa None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:33.554180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a2264-cbaa-4d8d-80b1-3fc7649a3c6b None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:33.555200 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 46/91] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:39:33 2022] GET /identity/v3/users/f6118de0629f4d08aa5388377f28b7ac => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:33.563142 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129ffcf-ec1f-4697-b26f-1eba210037aa None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:33.605171 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:33.678441 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a2264-cbaa-4d8d-80b1-3fc7649a3c6b None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:33.721147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21be99-1ba3-4637-a656-4e8af47648e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:33.722128 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21be99-1ba3-4637-a656-4e8af47648e9 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:33.722427 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21be99-1ba3-4637-a656-4e8af47648e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:33.722740 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21be99-1ba3-4637-a656-4e8af47648e9 None admin] PATH_INFO: `/v3/projects/fde6b97a4d09416a81257067f1d1b919` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:33.723974 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb21be99-1ba3-4637-a656-4e8af47648e9 None admin] RBAC: Authorizing `identity:get_project(project_id=fde6b97a4d09416a81257067f1d1b919)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:33.725630 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb21be99-1ba3-4637-a656-4e8af47648e9 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:33.734899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 46/92] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:39:33 2022] GET /identity/v3/projects/fde6b97a4d09416a81257067f1d1b919 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:33.735854 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21be99-1ba3-4637-a656-4e8af47648e9 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:33.740147 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129ffcf-ec1f-4697-b26f-1eba210037aa None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:33.782998 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d42913-36ca-4368-ac25-235479d2b7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:33.784070 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d42913-36ca-4368-ac25-235479d2b7d0 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:33.784552 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d42913-36ca-4368-ac25-235479d2b7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:33.785051 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d42913-36ca-4368-ac25-235479d2b7d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:33.857910 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:33.871230 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d42913-36ca-4368-ac25-235479d2b7d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:33.873511 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d42913-36ca-4368-ac25-235479d2b7d0 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:33.898202 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:33.909479 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21be99-1ba3-4637-a656-4e8af47648e9 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:33.981062 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e026c-69b5-42a9-b9d3-998a96817fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:33.982090 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e026c-69b5-42a9-b9d3-998a96817fff None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:33.982522 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e026c-69b5-42a9-b9d3-998a96817fff None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:33.982946 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e026c-69b5-42a9-b9d3-998a96817fff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:33.983946 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41e026c-69b5-42a9-b9d3-998a96817fff None admin] RBAC: Adding query filter params (role.id=a47f5df8505f49419d0acb4f0f287610, scope.project.id=fde6b97a4d09416a81257067f1d1b919, user.id=f6118de0629f4d08aa5388377f28b7ac) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:33.984406 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41e026c-69b5-42a9-b9d3-998a96817fff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:33.986158 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41e026c-69b5-42a9-b9d3-998a96817fff None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:33.999614 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 47/93] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:39:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 17 16:39:34.000735 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d42913-36ca-4368-ac25-235479d2b7d0 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:34.001757 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 47/94] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:39:33 2022] GET /identity/v3/role_assignments?user.id=f6118de0629f4d08aa5388377f28b7ac&scope.project.id=fde6b97a4d09416a81257067f1d1b919&role.id=a47f5df8505f49419d0acb4f0f287610 => generated 259 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:34.012849 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e026c-69b5-42a9-b9d3-998a96817fff None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:34.162652 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e026c-69b5-42a9-b9d3-998a96817fff None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:34.205897 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:34.220235 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d42913-36ca-4368-ac25-235479d2b7d0 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:34.238255 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170e48c-6fd4-4e82-8a12-45faf6c0868e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:34.239753 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170e48c-6fd4-4e82-8a12-45faf6c0868e None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:34.240045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170e48c-6fd4-4e82-8a12-45faf6c0868e None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:34.240328 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170e48c-6fd4-4e82-8a12-45faf6c0868e None admin] PATH_INFO: `/v3/projects/1732be3d65f741438a4c11098484f9e0` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:34.245729 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e170e48c-6fd4-4e82-8a12-45faf6c0868e None admin] RBAC: Authorizing `identity:get_project(project_id=1732be3d65f741438a4c11098484f9e0)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:34.247374 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e170e48c-6fd4-4e82-8a12-45faf6c0868e None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:34.249608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 48/95] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:39:34 2022] GET /identity/v3/projects/1732be3d65f741438a4c11098484f9e0 => generated 312 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:34.251034 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:34.257278 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170e48c-6fd4-4e82-8a12-45faf6c0868e None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:34.284914 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05c8167-ab50-4bd7-8b9a-ee7381d0174c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:34.286567 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05c8167-ab50-4bd7-8b9a-ee7381d0174c None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:34.287176 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05c8167-ab50-4bd7-8b9a-ee7381d0174c None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:34.287685 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05c8167-ab50-4bd7-8b9a-ee7381d0174c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:34.288768 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05c8167-ab50-4bd7-8b9a-ee7381d0174c None admin] RBAC: Adding query filter params (role.id=71cf67eb04704663aff719f6e3804f1a, scope.project.id=fde6b97a4d09416a81257067f1d1b919, user.id=f6118de0629f4d08aa5388377f28b7ac) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:34.289355 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05c8167-ab50-4bd7-8b9a-ee7381d0174c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:34.291052 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05c8167-ab50-4bd7-8b9a-ee7381d0174c None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:34.298308 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 48/96] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:39:33 2022] GET /identity/v3/role_assignments?user.id=f6118de0629f4d08aa5388377f28b7ac&scope.project.id=fde6b97a4d09416a81257067f1d1b919&role.id=71cf67eb04704663aff719f6e3804f1a => generated 259 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:34.434926 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170e48c-6fd4-4e82-8a12-45faf6c0868e None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:34.471678 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e917b-dbaf-4f19-a9b9-675cf823a619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:34.472361 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e917b-dbaf-4f19-a9b9-675cf823a619 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:34.472676 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e917b-dbaf-4f19-a9b9-675cf823a619 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:34.473115 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e917b-dbaf-4f19-a9b9-675cf823a619 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:34.474145 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e917b-dbaf-4f19-a9b9-675cf823a619 None admin] RBAC: Adding query filter params (role.id=a47f5df8505f49419d0acb4f0f287610, scope.project.id=1732be3d65f741438a4c11098484f9e0, user.id=f6118de0629f4d08aa5388377f28b7ac) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:34.474479 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e917b-dbaf-4f19-a9b9-675cf823a619 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:34.475975 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e917b-dbaf-4f19-a9b9-675cf823a619 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:34.482534 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 49/97] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:39:34 2022] GET /identity/v3/role_assignments?user.id=f6118de0629f4d08aa5388377f28b7ac&scope.project.id=1732be3d65f741438a4c11098484f9e0&role.id=a47f5df8505f49419d0acb4f0f287610 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:35.631280 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ee513-d4ba-4a78-98e4-35de253f8234 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:35.632307 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ee513-d4ba-4a78-98e4-35de253f8234 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:35.632866 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ee513-d4ba-4a78-98e4-35de253f8234 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:35.634720 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 49/98] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:35.646794 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a203f-b674-45cb-a6bc-f56e2164976d None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:35.650517 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a203f-b674-45cb-a6bc-f56e2164976d None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:35.650517 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a203f-b674-45cb-a6bc-f56e2164976d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:35.672944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-8d1a203f-b674-45cb-a6bc-f56e2164976d None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:35.674239 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-8d1a203f-b674-45cb-a6bc-f56e2164976d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:35.745252 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:35.774299 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 50/99] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:39:35.810155 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ee513-d4ba-4a78-98e4-35de253f8234 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:35.826499 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34abdd46-e6f9-40b0-b567-ca7b9c828032 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:35.826793 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34abdd46-e6f9-40b0-b567-ca7b9c828032 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:35.827082 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34abdd46-e6f9-40b0-b567-ca7b9c828032 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:35.828481 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 51/100] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:35.838316 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0ea1f-cea7-4462-999b-1de4ceed77cf None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:35.838603 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0ea1f-cea7-4462-999b-1de4ceed77cf None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:35.839737 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0ea1f-cea7-4462-999b-1de4ceed77cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:35.871713 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-1ff0ea1f-cea7-4462-999b-1de4ceed77cf None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:35.873981 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-1ff0ea1f-cea7-4462-999b-1de4ceed77cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:36.026290 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 52/101] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:39:36.066285 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ee513-d4ba-4a78-98e4-35de253f8234 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:36.070738 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff0ea1f-cea7-4462-999b-1de4ceed77cf None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:36.133695 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac71f72-c40c-4ecb-9bca-8f142d3e8f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:36.134715 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac71f72-c40c-4ecb-9bca-8f142d3e8f74 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:36.135107 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac71f72-c40c-4ecb-9bca-8f142d3e8f74 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:36.135509 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac71f72-c40c-4ecb-9bca-8f142d3e8f74 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:36.160805 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac71f72-c40c-4ecb-9bca-8f142d3e8f74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:39:36.187130 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-7ac71f72-c40c-4ecb-9bca-8f142d3e8f74 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 17 16:39:36.187130 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:39:36.187130 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:39:36.187130 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:39:36.187130 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:39:36.187130 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:39:36.187130 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:39:36.187130 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:39:36.187130 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:39:36.187130 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:39:36.187130 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:39:36.187130 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:39:36.187130 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:39:36.187130 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:39:36.187130 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:39:36.187130 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:39:36.187970 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:39:36.187970 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:39:36.187970 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:39:36.187970 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:39:36.187970 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:39:36.187970 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:39:36.187970 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:39:36.187970 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:39:36.187970 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:39:36.187970 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:39:36.187970 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:39:36.187970 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:39:36.187970 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:39:36.187970 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:39:36.187970 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:39:36.187970 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:39:36.189240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:39:36.189240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:39:36.189240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:39:36.189240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:39:36.189240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:39:36.189240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:39:36.189240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:39:36.189240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:39:36.189240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:39:36.189240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:39:36.189240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 17 16:39:36.189240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:39:36.189240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 50/102] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:39:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:39:36.198641 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac71f72-c40c-4ecb-9bca-8f142d3e8f74 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:36.242438 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff0ea1f-cea7-4462-999b-1de4ceed77cf None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:36.259736 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:36.282090 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13731dd3-e801-4083-b924-8ed0a07b1c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:36.282827 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13731dd3-e801-4083-b924-8ed0a07b1c9e None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:36.283117 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13731dd3-e801-4083-b924-8ed0a07b1c9e None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:36.283402 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13731dd3-e801-4083-b924-8ed0a07b1c9e None admin] PATH_INFO: `/v3/domains/default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:36.284662 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13731dd3-e801-4083-b924-8ed0a07b1c9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:36.286796 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13731dd3-e801-4083-b924-8ed0a07b1c9e None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:36.288625 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 53/103] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:39:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:36.289885 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ae2ea-c538-444d-933e-cf0973fde59f None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:36.290180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ae2ea-c538-444d-933e-cf0973fde59f None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:36.290462 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ae2ea-c538-444d-933e-cf0973fde59f None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:36.291827 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 54/104] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:36.298394 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b5f22-87e5-427a-b165-5be513f57a2e None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:36.298687 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b5f22-87e5-427a-b165-5be513f57a2e None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:36.298971 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b5f22-87e5-427a-b165-5be513f57a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:36.326434 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:36.344533 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-034b5f22-87e5-427a-b165-5be513f57a2e None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:36.346532 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-034b5f22-87e5-427a-b165-5be513f57a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:36.437364 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac71f72-c40c-4ecb-9bca-8f142d3e8f74 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:36.508473 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 55/105] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 17 16:39:36.510224 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b5f22-87e5-427a-b165-5be513f57a2e None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:36.515242 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052bc97a-c900-47f4-8c3f-e9ff5e23cb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:36.516359 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bc97a-c900-47f4-8c3f-e9ff5e23cb03 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:36.517591 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bc97a-c900-47f4-8c3f-e9ff5e23cb03 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:36.518331 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052bc97a-c900-47f4-8c3f-e9ff5e23cb03 None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:36.519325 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052bc97a-c900-47f4-8c3f-e9ff5e23cb03 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:36.524162 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052bc97a-c900-47f4-8c3f-e9ff5e23cb03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:36.525847 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052bc97a-c900-47f4-8c3f-e9ff5e23cb03 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:36.542907 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 51/106] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 17 16:39:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:36.549113 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052bc97a-c900-47f4-8c3f-e9ff5e23cb03 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:36.726560 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b5f22-87e5-427a-b165-5be513f57a2e None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:36.752017 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:36.757266 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052bc97a-c900-47f4-8c3f-e9ff5e23cb03 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:36.759456 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:36.799716 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b3224-3d39-49aa-8eb1-1ea8d096f7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:36.802690 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b3224-3d39-49aa-8eb1-1ea8d096f7e5 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:36.803186 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b3224-3d39-49aa-8eb1-1ea8d096f7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:36.805611 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b3224-3d39-49aa-8eb1-1ea8d096f7e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:36.825513 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0b3224-3d39-49aa-8eb1-1ea8d096f7e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:39:36.833484 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f46612-3f3e-480a-ab4f-1cd68dd389e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:36.834541 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f46612-3f3e-480a-ab4f-1cd68dd389e3 None admin] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:36.835085 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f46612-3f3e-480a-ab4f-1cd68dd389e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:36.835584 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f46612-3f3e-480a-ab4f-1cd68dd389e3 None admin] PATH_INFO: `/v3/projects` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:36.836516 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f46612-3f3e-480a-ab4f-1cd68dd389e3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:36.838278 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f46612-3f3e-480a-ab4f-1cd68dd389e3 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:36.847317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-eb0b3224-3d39-49aa-8eb1-1ea8d096f7e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:39:36.847317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:39:36.847317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:39:36.847317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:39:36.847317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:39:36.847317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:39:36.847317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:39:36.847317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:39:36.847317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:39:36.847317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:39:36.847317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:39:36.847317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:39:36.847317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:39:36.847317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:39:36.847317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:39:36.847317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:39:36.848219 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:39:36.848219 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:39:36.848219 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:39:36.848219 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:39:36.848219 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:39:36.848219 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:39:36.848219 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:39:36.848219 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:39:36.848219 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:39:36.848219 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:39:36.848219 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:39:36.848219 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:39:36.848219 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:39:36.848219 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:39:36.848219 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:39:36.848219 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:39:36.849000 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:39:36.849000 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:39:36.849000 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:39:36.849000 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:39:36.849000 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:39:36.849000 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:39:36.849000 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:39:36.849000 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:39:36.849000 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:39:36.849000 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:39:36.849000 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:39:36.849000 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:39:36.854173 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 52/107] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:39:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:39:36.856687 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b3224-3d39-49aa-8eb1-1ea8d096f7e5 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:36.973992 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 56/108] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Jul 17 16:39:36 2022] POST /identity/v3/projects => generated 302 bytes in 464 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:39:36.975180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f46612-3f3e-480a-ab4f-1cd68dd389e3 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:37.060192 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b3224-3d39-49aa-8eb1-1ea8d096f7e5 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:37.153527 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850287a6-b190-4194-b651-89ed39f23fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:37.154892 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850287a6-b190-4194-b651-89ed39f23fc0 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:37.155403 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850287a6-b190-4194-b651-89ed39f23fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:37.155886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850287a6-b190-4194-b651-89ed39f23fc0 None admin] PATH_INFO: `/v3/users/9eb540242e364ea7955ea951382ed935` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:37.201473 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850287a6-b190-4194-b651-89ed39f23fc0 None admin] RBAC: Authorizing `identity:get_user(user_id=9eb540242e364ea7955ea951382ed935)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:37.204355 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850287a6-b190-4194-b651-89ed39f23fc0 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:37.212469 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 53/109] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:39:36 2022] GET /identity/v3/users/9eb540242e364ea7955ea951382ed935 => generated 289 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:37.221471 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920e53c-bd03-4461-bb13-390402769fbf None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:37.221994 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920e53c-bd03-4461-bb13-390402769fbf None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:37.223822 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920e53c-bd03-4461-bb13-390402769fbf None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:37.226559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 54/110] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:37 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:37.228248 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920e53c-bd03-4461-bb13-390402769fbf None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:37.284544 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f46612-3f3e-480a-ab4f-1cd68dd389e3 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:37.301924 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:37.347261 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a5a0c-01ed-4a10-b821-4dd57a4d9b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:37.348850 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a5a0c-01ed-4a10-b821-4dd57a4d9b9c None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:37.349332 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a5a0c-01ed-4a10-b821-4dd57a4d9b9c None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:37.349854 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a5a0c-01ed-4a10-b821-4dd57a4d9b9c None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:37.351161 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1a5a0c-01ed-4a10-b821-4dd57a4d9b9c None admin] RBAC: Adding query filter params (name=admin) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:37.351742 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1a5a0c-01ed-4a10-b821-4dd57a4d9b9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:37.354286 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1a5a0c-01ed-4a10-b821-4dd57a4d9b9c None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:37.368626 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 57/111] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:39:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:37.370187 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273f25a-c659-43f8-994f-7022feaa786e None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:37.370618 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273f25a-c659-43f8-994f-7022feaa786e None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:37.371099 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273f25a-c659-43f8-994f-7022feaa786e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:37.396593 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-b273f25a-c659-43f8-994f-7022feaa786e None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:37.398602 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-b273f25a-c659-43f8-994f-7022feaa786e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:37.415486 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920e53c-bd03-4461-bb13-390402769fbf None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:37.429511 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:37.457781 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574a90e1-046a-452b-8f0d-1519317086e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:37.458479 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a90e1-046a-452b-8f0d-1519317086e8 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:37.458773 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a90e1-046a-452b-8f0d-1519317086e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:37.459058 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a90e1-046a-452b-8f0d-1519317086e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:37.499916 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 58/112] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:39:37.501645 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500bca3-ec53-454a-8020-cba703aa7c85 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:37.502036 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500bca3-ec53-454a-8020-cba703aa7c85 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:37.502453 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500bca3-ec53-454a-8020-cba703aa7c85 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:37.505071 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 59/113] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:37.506513 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500bca3-ec53-454a-8020-cba703aa7c85 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:37.541789 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574a90e1-046a-452b-8f0d-1519317086e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:37.543872 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574a90e1-046a-452b-8f0d-1519317086e8 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:37.568688 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:37.626046 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500bca3-ec53-454a-8020-cba703aa7c85 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:37.655454 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 55/114] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:39:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 17 16:39:37.657016 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed978e1d-af74-4c49-8dd8-377bd674eb76 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:37.657439 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed978e1d-af74-4c49-8dd8-377bd674eb76 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:37.657873 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed978e1d-af74-4c49-8dd8-377bd674eb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:37.670066 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6787fbf-2530-4e3a-a5b9-4e8f1294d258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:37.671484 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6787fbf-2530-4e3a-a5b9-4e8f1294d258 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:37.671484 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6787fbf-2530-4e3a-a5b9-4e8f1294d258 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:37.671912 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6787fbf-2530-4e3a-a5b9-4e8f1294d258 None admin] PATH_INFO: `/v3/users/45e7917424634d8897c1eb02702d4bc2` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:37.673289 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6787fbf-2530-4e3a-a5b9-4e8f1294d258 None admin] RBAC: Authorizing `identity:get_user(user_id=45e7917424634d8897c1eb02702d4bc2)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:37.675121 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6787fbf-2530-4e3a-a5b9-4e8f1294d258 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:37.677833 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-ed978e1d-af74-4c49-8dd8-377bd674eb76 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:37.678095 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 60/115] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:39:37 2022] GET /identity/v3/users/45e7917424634d8897c1eb02702d4bc2 => generated 247 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:37.679014 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6787fbf-2530-4e3a-a5b9-4e8f1294d258 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:37.679557 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-ed978e1d-af74-4c49-8dd8-377bd674eb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:37.705281 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:37.755615 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 56/116] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:39:37.756711 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed978e1d-af74-4c49-8dd8-377bd674eb76 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:37.797010 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6787fbf-2530-4e3a-a5b9-4e8f1294d258 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:37.811538 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:37.839713 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5d7e5-192d-4c97-92da-0e0119d564d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:37.840740 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5d7e5-192d-4c97-92da-0e0119d564d6 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:37.841146 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5d7e5-192d-4c97-92da-0e0119d564d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:37.841525 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5d7e5-192d-4c97-92da-0e0119d564d6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:37.853509 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e5d7e5-192d-4c97-92da-0e0119d564d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:39:37.863322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-98e5d7e5-192d-4c97-92da-0e0119d564d6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 17 16:39:37.863322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:39:37.863322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:39:37.863322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:39:37.863322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:39:37.863322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:39:37.863322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:39:37.863322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:39:37.863322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:39:37.863322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:39:37.863322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:39:37.863322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:39:37.863322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:39:37.863322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:39:37.863322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:39:37.863322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:39:37.864233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:39:37.864233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:39:37.864233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:39:37.864233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:39:37.864233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:39:37.864233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:39:37.864233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:39:37.864233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:39:37.864233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:39:37.864233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:39:37.864233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:39:37.864233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:39:37.864233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:39:37.864233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:39:37.864233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:39:37.864233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:39:37.865022 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:39:37.865022 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:39:37.865022 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:39:37.865022 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:39:37.865022 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:39:37.865022 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:39:37.865022 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:39:37.865022 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:39:37.865022 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:39:37.865022 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:39:37.865022 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 17 16:39:37.865022 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:39:37.865602 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 61/117] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 17 16:39:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:39:37.866471 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5d7e5-192d-4c97-92da-0e0119d564d6 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:37.870816 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed978e1d-af74-4c49-8dd8-377bd674eb76 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:37.914393 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c703fb2b-e92b-43bf-b4de-de56326b1188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:37.914393 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703fb2b-e92b-43bf-b4de-de56326b1188 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:37.914393 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703fb2b-e92b-43bf-b4de-de56326b1188 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:37.914393 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703fb2b-e92b-43bf-b4de-de56326b1188 None admin] PATH_INFO: `/v3/projects/fde6b97a4d09416a81257067f1d1b919` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:37.914897 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c703fb2b-e92b-43bf-b4de-de56326b1188 None admin] RBAC: Authorizing `identity:get_project(project_id=fde6b97a4d09416a81257067f1d1b919)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:37.914897 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c703fb2b-e92b-43bf-b4de-de56326b1188 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:37.916845 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 57/118] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:39:37 2022] GET /identity/v3/projects/fde6b97a4d09416a81257067f1d1b919 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:37.917212 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c703fb2b-e92b-43bf-b4de-de56326b1188 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:37.972035 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:37.997265 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5d7e5-192d-4c97-92da-0e0119d564d6 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:38.044697 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a70928-2cbc-4c77-93a9-ccd569123b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:38.045628 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a70928-2cbc-4c77-93a9-ccd569123b98 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:38.045917 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a70928-2cbc-4c77-93a9-ccd569123b98 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:38.046242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a70928-2cbc-4c77-93a9-ccd569123b98 None admin] PATH_INFO: `/v3/projects/fde6b97a4d09416a81257067f1d1b919` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:38.047545 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a70928-2cbc-4c77-93a9-ccd569123b98 None admin] RBAC: Authorizing `identity:get_project(project_id=fde6b97a4d09416a81257067f1d1b919)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:38.049200 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a70928-2cbc-4c77-93a9-ccd569123b98 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:38.051407 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c703fb2b-e92b-43bf-b4de-de56326b1188 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:38.051867 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 62/119] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:39:37 2022] GET /identity/v3/projects/fde6b97a4d09416a81257067f1d1b919 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:38.052995 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a70928-2cbc-4c77-93a9-ccd569123b98 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:38.076478 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:38.094946 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb24fa8-2f46-4817-917e-23a71e365de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:38.095762 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb24fa8-2f46-4817-917e-23a71e365de4 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:38.096053 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb24fa8-2f46-4817-917e-23a71e365de4 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:38.096341 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb24fa8-2f46-4817-917e-23a71e365de4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:38.109799 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb24fa8-2f46-4817-917e-23a71e365de4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:39:38.119092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-4fb24fa8-2f46-4817-917e-23a71e365de4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:39:38.119092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:39:38.119092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:39:38.119092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:39:38.119092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:39:38.119092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:39:38.119092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:39:38.119092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:39:38.119092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:39:38.119092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:39:38.119092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:39:38.119092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:39:38.119092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:39:38.119092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:39:38.119092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:39:38.119092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:39:38.120222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:39:38.120222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:39:38.120222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:39:38.120222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:39:38.120222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:39:38.120222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:39:38.120222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:39:38.120222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:39:38.120222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:39:38.120222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:39:38.120222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:39:38.120222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:39:38.120222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:39:38.120222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:39:38.120222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:39:38.120222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:39:38.121069 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:39:38.121069 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:39:38.121069 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:39:38.121069 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:39:38.121069 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:39:38.121069 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:39:38.121069 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:39:38.121069 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:39:38.121069 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:39:38.121069 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:39:38.121069 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:39:38.121069 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:39:38.125092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 58/120] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:39:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:39:38.126430 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdcd19-2bc6-4219-a108-ab3b52718883 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:38.126726 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdcd19-2bc6-4219-a108-ab3b52718883 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:38.127012 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdcd19-2bc6-4219-a108-ab3b52718883 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:38.129373 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 59/121] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:38.130729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbdcd19-2bc6-4219-a108-ab3b52718883 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:38.188179 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a70928-2cbc-4c77-93a9-ccd569123b98 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:38.232218 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f9e732-87fc-4557-9c52-3c52ff165e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:38.232942 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9e732-87fc-4557-9c52-3c52ff165e12 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:38.233224 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9e732-87fc-4557-9c52-3c52ff165e12 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:38.233599 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9e732-87fc-4557-9c52-3c52ff165e12 None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:38.234289 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f9e732-87fc-4557-9c52-3c52ff165e12 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:38.234601 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f9e732-87fc-4557-9c52-3c52ff165e12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:38.236040 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f9e732-87fc-4557-9c52-3c52ff165e12 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:38.245981 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 63/122] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:39:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:38.251648 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f9e732-87fc-4557-9c52-3c52ff165e12 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:38.260300 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbdcd19-2bc6-4219-a108-ab3b52718883 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:38.266245 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:38.301550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed5513-d9ed-4280-a6cd-48954a29a230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:38.302339 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed5513-d9ed-4280-a6cd-48954a29a230 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:38.302699 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed5513-d9ed-4280-a6cd-48954a29a230 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:38.302986 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed5513-d9ed-4280-a6cd-48954a29a230 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:38.303836 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ed5513-d9ed-4280-a6cd-48954a29a230 None admin] RBAC: Adding query filter params (role.id=e7249313e5334765ba16793745caed63, scope.project.id=fde6b97a4d09416a81257067f1d1b919, user.id=9eb540242e364ea7955ea951382ed935) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:38.304153 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ed5513-d9ed-4280-a6cd-48954a29a230 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:38.305624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ed5513-d9ed-4280-a6cd-48954a29a230 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:38.312114 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 60/123] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:39:38 2022] GET /identity/v3/role_assignments?user.id=9eb540242e364ea7955ea951382ed935&scope.project.id=fde6b97a4d09416a81257067f1d1b919&role.id=e7249313e5334765ba16793745caed63 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:38.318396 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed5513-d9ed-4280-a6cd-48954a29a230 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:38.374177 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:38.387679 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f9e732-87fc-4557-9c52-3c52ff165e12 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:38.441305 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05af3a-d143-4d2d-bcdb-80266b6629d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:38.442945 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05af3a-d143-4d2d-bcdb-80266b6629d1 None admin] REQUEST_METHOD: `PUT` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:38.443242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05af3a-d143-4d2d-bcdb-80266b6629d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:38.443527 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05af3a-d143-4d2d-bcdb-80266b6629d1 None admin] PATH_INFO: `/v3/projects/fde6b97a4d09416a81257067f1d1b919/users/45e7917424634d8897c1eb02702d4bc2/roles/1025051975f64f689effa58dc4259e36` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:38.447136 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd05af3a-d143-4d2d-bcdb-80266b6629d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=fde6b97a4d09416a81257067f1d1b919, user_id=45e7917424634d8897c1eb02702d4bc2, role_id=1025051975f64f689effa58dc4259e36)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:38.448996 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd05af3a-d143-4d2d-bcdb-80266b6629d1 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:38.450537 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed5513-d9ed-4280-a6cd-48954a29a230 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:38.492528 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd29f8-472f-445a-ad99-735a4d81fc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:38.493602 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd29f8-472f-445a-ad99-735a4d81fc91 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:38.493894 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd29f8-472f-445a-ad99-735a4d81fc91 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:38.494179 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd29f8-472f-445a-ad99-735a4d81fc91 None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:38.494943 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd29f8-472f-445a-ad99-735a4d81fc91 None admin] RBAC: Adding query filter params (name=member) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:38.495289 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd29f8-472f-445a-ad99-735a4d81fc91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:38.496690 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd29f8-472f-445a-ad99-735a4d81fc91 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:38.509947 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 61/124] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 17 16:39:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:38.509947 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5c0b8-8850-4edb-8dda-e86c7b989cca None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:38.509947 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5c0b8-8850-4edb-8dda-e86c7b989cca None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:38.509947 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5c0b8-8850-4edb-8dda-e86c7b989cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:38.527775 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-49c5c0b8-8850-4edb-8dda-e86c7b989cca None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:38.529092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-49c5c0b8-8850-4edb-8dda-e86c7b989cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:38.534749 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 64/125] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 17 16:39:38 2022] PUT /identity/v3/projects/fde6b97a4d09416a81257067f1d1b919/users/45e7917424634d8897c1eb02702d4bc2/roles/1025051975f64f689effa58dc4259e36 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 17 16:39:38.538564 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05af3a-d143-4d2d-bcdb-80266b6629d1 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:38.624256 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 62/126] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:39:38.625273 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c5c0b8-8850-4edb-8dda-e86c7b989cca None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:38.640001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:38.702976 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:38.708493 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05af3a-d143-4d2d-bcdb-80266b6629d1 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:38.787768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5a120-ccb9-4031-8215-aa857cc17d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:38.788677 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5a120-ccb9-4031-8215-aa857cc17d7b None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:38.788996 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5a120-ccb9-4031-8215-aa857cc17d7b None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:38.789289 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5a120-ccb9-4031-8215-aa857cc17d7b None admin] PATH_INFO: `/v3/users/f6118de0629f4d08aa5388377f28b7ac` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:38.793652 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d5a120-ccb9-4031-8215-aa857cc17d7b None admin] RBAC: Authorizing `identity:get_user(user_id=f6118de0629f4d08aa5388377f28b7ac)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:38.795428 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d5a120-ccb9-4031-8215-aa857cc17d7b None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:38.796304 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c5c0b8-8850-4edb-8dda-e86c7b989cca None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:38.797613 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 65/127] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:39:38 2022] GET /identity/v3/users/f6118de0629f4d08aa5388377f28b7ac => generated 275 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:38.799268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5a120-ccb9-4031-8215-aa857cc17d7b None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:38.840897 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140ec2d-8ceb-4cf1-ab21-d47e0d6708fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:38.841787 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140ec2d-8ceb-4cf1-ab21-d47e0d6708fd None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:38.842072 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140ec2d-8ceb-4cf1-ab21-d47e0d6708fd None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:38.842349 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140ec2d-8ceb-4cf1-ab21-d47e0d6708fd None admin] PATH_INFO: `/v3/users/f6118de0629f4d08aa5388377f28b7ac` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:38.843580 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140ec2d-8ceb-4cf1-ab21-d47e0d6708fd None admin] RBAC: Authorizing `identity:get_user(user_id=f6118de0629f4d08aa5388377f28b7ac)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:38.845130 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140ec2d-8ceb-4cf1-ab21-d47e0d6708fd None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:38.847097 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 63/128] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:39:38 2022] GET /identity/v3/users/f6118de0629f4d08aa5388377f28b7ac => generated 275 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:38.852040 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140ec2d-8ceb-4cf1-ab21-d47e0d6708fd None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:38.934136 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5a120-ccb9-4031-8215-aa857cc17d7b None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:38.996023 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140ec2d-8ceb-4cf1-ab21-d47e0d6708fd None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:38.998112 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:39.038737 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad176f-044c-4d70-aade-8d82238cc222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:39.040106 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad176f-044c-4d70-aade-8d82238cc222 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:39.040670 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad176f-044c-4d70-aade-8d82238cc222 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:39.041208 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad176f-044c-4d70-aade-8d82238cc222 None admin] PATH_INFO: `/v3/projects/fde6b97a4d09416a81257067f1d1b919` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:39.042736 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ad176f-044c-4d70-aade-8d82238cc222 None admin] RBAC: Authorizing `identity:get_project(project_id=fde6b97a4d09416a81257067f1d1b919)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:39.044592 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ad176f-044c-4d70-aade-8d82238cc222 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:39.046571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 64/129] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:39:38 2022] GET /identity/v3/projects/fde6b97a4d09416a81257067f1d1b919 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:39.046680 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0460b5-2fa4-4ac5-b146-e4232450991d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:39.047503 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0460b5-2fa4-4ac5-b146-e4232450991d None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:39.047793 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0460b5-2fa4-4ac5-b146-e4232450991d None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:39.048074 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0460b5-2fa4-4ac5-b146-e4232450991d None admin] PATH_INFO: `/v3/roles/member` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:39.048330 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad176f-044c-4d70-aade-8d82238cc222 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:39.065001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0460b5-2fa4-4ac5-b146-e4232450991d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:39:39.074725 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:39.079431 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-7a0460b5-2fa4-4ac5-b146-e4232450991d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:39:39.079431 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:39:39.079431 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:39:39.079431 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:39:39.079431 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:39:39.079431 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:39:39.079431 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:39:39.079431 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:39:39.079431 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:39:39.079431 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:39:39.079431 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:39:39.079431 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:39:39.079431 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:39:39.079431 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:39:39.079431 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:39:39.079431 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:39:39.080279 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:39:39.080279 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:39:39.080279 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:39:39.080279 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:39:39.080279 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:39:39.080279 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:39:39.080279 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:39:39.080279 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:39:39.080279 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:39:39.080279 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:39:39.080279 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:39:39.080279 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:39:39.080279 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:39:39.080279 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:39:39.080279 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:39:39.080279 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:39:39.081073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:39:39.081073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:39:39.081073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:39:39.081073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:39:39.081073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:39:39.081073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:39:39.081073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:39:39.081073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:39:39.081073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:39:39.081073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:39:39.081073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:39:39.081073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:39:39.081073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 66/130] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:39:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:39:39.084071 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0460b5-2fa4-4ac5-b146-e4232450991d None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:39.251115 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ad176f-044c-4d70-aade-8d82238cc222 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:39.278458 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0460b5-2fa4-4ac5-b146-e4232450991d None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:39.308452 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5db89fd-da2a-45e3-81cd-cd21a6a9089d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:39.310481 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db89fd-da2a-45e3-81cd-cd21a6a9089d None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:39.310481 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db89fd-da2a-45e3-81cd-cd21a6a9089d None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:39.310481 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db89fd-da2a-45e3-81cd-cd21a6a9089d None admin] PATH_INFO: `/v3/projects/fde6b97a4d09416a81257067f1d1b919` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:39.311129 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5db89fd-da2a-45e3-81cd-cd21a6a9089d None admin] RBAC: Authorizing `identity:get_project(project_id=fde6b97a4d09416a81257067f1d1b919)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:39.312591 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5db89fd-da2a-45e3-81cd-cd21a6a9089d None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:39.314807 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 65/131] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:39:39 2022] GET /identity/v3/projects/fde6b97a4d09416a81257067f1d1b919 => generated 298 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:39.315612 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5db89fd-da2a-45e3-81cd-cd21a6a9089d None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:39.334579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f20d895-9607-4a0e-8a0c-c64ffaf53ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:39.336397 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f20d895-9607-4a0e-8a0c-c64ffaf53ce0 None admin] REQUEST_METHOD: `PUT` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:39.336963 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f20d895-9607-4a0e-8a0c-c64ffaf53ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:39.337444 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f20d895-9607-4a0e-8a0c-c64ffaf53ce0 None admin] PATH_INFO: `/v3/projects/fde6b97a4d09416a81257067f1d1b919/users/f6118de0629f4d08aa5388377f28b7ac/roles/71cf67eb04704663aff719f6e3804f1a` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:39.342712 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f20d895-9607-4a0e-8a0c-c64ffaf53ce0 None admin] RBAC: Authorizing `identity:create_grant(project_id=fde6b97a4d09416a81257067f1d1b919, user_id=f6118de0629f4d08aa5388377f28b7ac, role_id=71cf67eb04704663aff719f6e3804f1a)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:39.345310 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f20d895-9607-4a0e-8a0c-c64ffaf53ce0 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:39.389948 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:39.456024 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5db89fd-da2a-45e3-81cd-cd21a6a9089d None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:39.461175 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 67/132] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 17 16:39:39 2022] PUT /identity/v3/projects/fde6b97a4d09416a81257067f1d1b919/users/f6118de0629f4d08aa5388377f28b7ac/roles/71cf67eb04704663aff719f6e3804f1a => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 17 16:39:39.464492 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f20d895-9607-4a0e-8a0c-c64ffaf53ce0 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:39.553003 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:39.567187 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b609c-fe20-485b-a9f6-f95478082d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:39.569135 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b609c-fe20-485b-a9f6-f95478082d97 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:39.569135 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b609c-fe20-485b-a9f6-f95478082d97 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:39.569135 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b609c-fe20-485b-a9f6-f95478082d97 None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:39.569567 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b609c-fe20-485b-a9f6-f95478082d97 None admin] RBAC: Adding query filter params (name=member) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:39.569899 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b609c-fe20-485b-a9f6-f95478082d97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:39.571307 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b609c-fe20-485b-a9f6-f95478082d97 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:39.596432 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 66/133] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 17 16:39:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:39.600371 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14baba39-8b7c-45cb-b5c8-225ec4a9789c None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:39.600371 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14baba39-8b7c-45cb-b5c8-225ec4a9789c None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:39.600371 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14baba39-8b7c-45cb-b5c8-225ec4a9789c None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:39.600371 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 67/134] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:39.606280 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626e56b-9624-4731-83c0-32ada67ee9ad None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:39.606573 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626e56b-9624-4731-83c0-32ada67ee9ad None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:39.606866 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b626e56b-9624-4731-83c0-32ada67ee9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:39.615105 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f20d895-9607-4a0e-8a0c-c64ffaf53ce0 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:39.641963 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-b626e56b-9624-4731-83c0-32ada67ee9ad None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:39.643176 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-b626e56b-9624-4731-83c0-32ada67ee9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:39.657702 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9338546-a27b-428a-84d2-93db0b07451d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:39.660658 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9338546-a27b-428a-84d2-93db0b07451d None admin] REQUEST_METHOD: `PUT` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:39.661218 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9338546-a27b-428a-84d2-93db0b07451d None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:39.661718 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9338546-a27b-428a-84d2-93db0b07451d None admin] PATH_INFO: `/v3/projects/fde6b97a4d09416a81257067f1d1b919/users/f6118de0629f4d08aa5388377f28b7ac/roles/a47f5df8505f49419d0acb4f0f287610` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:39.664849 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9338546-a27b-428a-84d2-93db0b07451d None admin] RBAC: Authorizing `identity:create_grant(project_id=fde6b97a4d09416a81257067f1d1b919, user_id=f6118de0629f4d08aa5388377f28b7ac, role_id=a47f5df8505f49419d0acb4f0f287610)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:39.666881 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9338546-a27b-428a-84d2-93db0b07451d None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:39.728553 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 68/135] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:39:39.729522 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626e56b-9624-4731-83c0-32ada67ee9ad None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:39.754974 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 68/136] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 17 16:39:39 2022] PUT /identity/v3/projects/fde6b97a4d09416a81257067f1d1b919/users/f6118de0629f4d08aa5388377f28b7ac/roles/a47f5df8505f49419d0acb4f0f287610 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 17 16:39:39.766594 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9338546-a27b-428a-84d2-93db0b07451d None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:39.791042 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:39.947513 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b626e56b-9624-4731-83c0-32ada67ee9ad None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:39.995263 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9338546-a27b-428a-84d2-93db0b07451d None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:40.026015 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60711c3a-1952-4c47-9755-8aee47244499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:40.028150 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60711c3a-1952-4c47-9755-8aee47244499 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:40.030334 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60711c3a-1952-4c47-9755-8aee47244499 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:40.030828 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60711c3a-1952-4c47-9755-8aee47244499 None admin] PATH_INFO: `/v3/users/f6118de0629f4d08aa5388377f28b7ac` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:40.032541 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60711c3a-1952-4c47-9755-8aee47244499 None admin] RBAC: Authorizing `identity:get_user(user_id=f6118de0629f4d08aa5388377f28b7ac)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:40.036027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60711c3a-1952-4c47-9755-8aee47244499 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:40.042286 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:40.047346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 69/137] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:39:39 2022] GET /identity/v3/users/f6118de0629f4d08aa5388377f28b7ac => generated 275 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:40.053276 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60711c3a-1952-4c47-9755-8aee47244499 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:40.081598 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9edfd6b-2090-4852-afc1-22003a0abb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:40.084279 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9edfd6b-2090-4852-afc1-22003a0abb49 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:40.084980 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9edfd6b-2090-4852-afc1-22003a0abb49 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:40.086970 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9edfd6b-2090-4852-afc1-22003a0abb49 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:40.089338 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9edfd6b-2090-4852-afc1-22003a0abb49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:40.093925 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9edfd6b-2090-4852-afc1-22003a0abb49 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:40.103544 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 69/138] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:39:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:40.103544 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9edfd6b-2090-4852-afc1-22003a0abb49 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:40.234121 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9edfd6b-2090-4852-afc1-22003a0abb49 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:40.253157 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:40.267368 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60711c3a-1952-4c47-9755-8aee47244499 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:40.292870 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e3cc4-f7e5-4639-b585-58d0d6f4c84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:40.294047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e3cc4-f7e5-4639-b585-58d0d6f4c84f None admin] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:40.294543 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e3cc4-f7e5-4639-b585-58d0d6f4c84f None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:40.295046 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e3cc4-f7e5-4639-b585-58d0d6f4c84f None admin] PATH_INFO: `/v3/users` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:40.296465 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92e3cc4-f7e5-4639-b585-58d0d6f4c84f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:40.299331 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92e3cc4-f7e5-4639-b585-58d0d6f4c84f None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:40.378764 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73071a0-a51b-42eb-8a0f-5349b8c42e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:40.380269 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73071a0-a51b-42eb-8a0f-5349b8c42e31 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:40.380932 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73071a0-a51b-42eb-8a0f-5349b8c42e31 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:40.381447 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73071a0-a51b-42eb-8a0f-5349b8c42e31 None admin] PATH_INFO: `/v3/projects/1732be3d65f741438a4c11098484f9e0` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:40.385020 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73071a0-a51b-42eb-8a0f-5349b8c42e31 None admin] RBAC: Authorizing `identity:get_project(project_id=1732be3d65f741438a4c11098484f9e0)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:40.387833 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73071a0-a51b-42eb-8a0f-5349b8c42e31 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:40.391087 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 70/139] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:39:40 2022] GET /identity/v3/projects/1732be3d65f741438a4c11098484f9e0 => generated 312 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:40.397726 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73071a0-a51b-42eb-8a0f-5349b8c42e31 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:40.431928 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 70/140] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 17 16:39:40 2022] POST /identity/v3/users => generated 283 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:39:40.665863 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73071a0-a51b-42eb-8a0f-5349b8c42e31 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:40.750013 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80b559-93a5-4a65-857f-6a09a2991ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:40.751657 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80b559-93a5-4a65-857f-6a09a2991ad7 None admin] REQUEST_METHOD: `PUT` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:40.752166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80b559-93a5-4a65-857f-6a09a2991ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:40.752649 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80b559-93a5-4a65-857f-6a09a2991ad7 None admin] PATH_INFO: `/v3/projects/1732be3d65f741438a4c11098484f9e0/users/f6118de0629f4d08aa5388377f28b7ac/roles/a47f5df8505f49419d0acb4f0f287610` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:40.757301 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c80b559-93a5-4a65-857f-6a09a2991ad7 None admin] RBAC: Authorizing `identity:create_grant(project_id=1732be3d65f741438a4c11098484f9e0, user_id=f6118de0629f4d08aa5388377f28b7ac, role_id=a47f5df8505f49419d0acb4f0f287610)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:40.760030 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c80b559-93a5-4a65-857f-6a09a2991ad7 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:40.845793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 71/141] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 17 16:39:40 2022] PUT /identity/v3/projects/1732be3d65f741438a4c11098484f9e0/users/f6118de0629f4d08aa5388377f28b7ac/roles/a47f5df8505f49419d0acb4f0f287610 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 17 16:39:41.596276 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e07a3b-fecd-43f2-9f48-5e42eccbc557 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:41.596797 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e07a3b-fecd-43f2-9f48-5e42eccbc557 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:41.597826 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e07a3b-fecd-43f2-9f48-5e42eccbc557 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:41.600525 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 71/142] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:41.607881 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b13a1d-409c-4ccc-9b4c-602b282fda5a None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:41.612726 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b13a1d-409c-4ccc-9b4c-602b282fda5a None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:41.612726 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b13a1d-409c-4ccc-9b4c-602b282fda5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:41.631308 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5816a50-bf51-4d92-a9b1-e4c818c67727 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:41.631778 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5816a50-bf51-4d92-a9b1-e4c818c67727 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:41.632245 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5816a50-bf51-4d92-a9b1-e4c818c67727 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:41.634924 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 72/143] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:41.644100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-89b13a1d-409c-4ccc-9b4c-602b282fda5a None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:41.647341 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-89b13a1d-409c-4ccc-9b4c-602b282fda5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:41.653739 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537f293-20fe-4c58-a358-dab95d0770d5 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:41.654368 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537f293-20fe-4c58-a358-dab95d0770d5 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:41.654843 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537f293-20fe-4c58-a358-dab95d0770d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:41.694137 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:41.699271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-c537f293-20fe-4c58-a358-dab95d0770d5 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:41.701225 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-c537f293-20fe-4c58-a358-dab95d0770d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:41.841194 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:41.858540 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 72/144] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:39:41.894717 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b13a1d-409c-4ccc-9b4c-602b282fda5a None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:41.932819 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 73/145] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:39:42.004776 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537f293-20fe-4c58-a358-dab95d0770d5 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:42.069490 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b13a1d-409c-4ccc-9b4c-602b282fda5a None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:42.122081 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3bc22-ebec-4c3b-be91-8ed25020de77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:42.122839 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3bc22-ebec-4c3b-be91-8ed25020de77 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:42.123127 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3bc22-ebec-4c3b-be91-8ed25020de77 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:42.123409 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3bc22-ebec-4c3b-be91-8ed25020de77 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:42.141985 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3bc22-ebec-4c3b-be91-8ed25020de77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:39:42.151021 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-fad3bc22-ebec-4c3b-be91-8ed25020de77 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 17 16:39:42.151021 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:39:42.151021 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:39:42.151021 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:39:42.151021 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:39:42.151021 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:39:42.151021 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:39:42.151021 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:39:42.151021 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:39:42.151021 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:39:42.151021 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:39:42.151021 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:39:42.151021 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:39:42.151021 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:39:42.151021 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:39:42.151021 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:39:42.151887 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:39:42.151887 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:39:42.151887 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:39:42.151887 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:39:42.151887 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:39:42.151887 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:39:42.151887 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:39:42.151887 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:39:42.151887 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:39:42.151887 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:39:42.151887 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:39:42.151887 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:39:42.151887 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:39:42.151887 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:39:42.151887 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:39:42.151887 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:39:42.152633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:39:42.152633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:39:42.152633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:39:42.152633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:39:42.152633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:39:42.152633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:39:42.152633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:39:42.152633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:39:42.152633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:39:42.152633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:39:42.152633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 17 16:39:42.152633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:39:42.152633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 73/146] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:39:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:39:42.159615 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3bc22-ebec-4c3b-be91-8ed25020de77 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:42.192969 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:42.270366 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537f293-20fe-4c58-a358-dab95d0770d5 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:42.290799 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3bc22-ebec-4c3b-be91-8ed25020de77 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:42.320432 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea056f-29c0-4380-b9dc-2633f7a79e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:42.321923 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea056f-29c0-4380-b9dc-2633f7a79e83 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:42.322428 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea056f-29c0-4380-b9dc-2633f7a79e83 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:42.322915 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea056f-29c0-4380-b9dc-2633f7a79e83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:42.335377 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1cea2d-0a2b-4746-a20f-4bd5e8397718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:42.336059 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cea2d-0a2b-4746-a20f-4bd5e8397718 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:42.336346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cea2d-0a2b-4746-a20f-4bd5e8397718 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:42.336626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cea2d-0a2b-4746-a20f-4bd5e8397718 None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:42.339675 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1cea2d-0a2b-4746-a20f-4bd5e8397718 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:42.339994 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1cea2d-0a2b-4746-a20f-4bd5e8397718 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:42.341429 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1cea2d-0a2b-4746-a20f-4bd5e8397718 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:42.346682 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddea056f-29c0-4380-b9dc-2633f7a79e83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:39:42.361177 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 74/147] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 17 16:39:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:42.366015 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-ddea056f-29c0-4380-b9dc-2633f7a79e83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:39:42.366015 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:39:42.366015 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:39:42.366015 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:39:42.366015 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:39:42.366015 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:39:42.366015 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:39:42.366015 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:39:42.366015 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:39:42.366015 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:39:42.366015 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:39:42.366015 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:39:42.366015 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:39:42.366015 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:39:42.366015 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:39:42.366015 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:39:42.366886 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:39:42.366886 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:39:42.366886 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:39:42.366886 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:39:42.366886 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:39:42.366886 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:39:42.366886 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:39:42.366886 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:39:42.366886 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:39:42.366886 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:39:42.366886 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:39:42.366886 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:39:42.366886 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:39:42.366886 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:39:42.366886 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:39:42.366886 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:39:42.367656 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:39:42.367656 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:39:42.367656 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:39:42.367656 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:39:42.367656 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:39:42.367656 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:39:42.367656 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:39:42.367656 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:39:42.367656 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:39:42.367656 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:39:42.367656 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:39:42.367656 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:39:42.367656 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 74/148] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:39:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:39:42.370395 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1cea2d-0a2b-4746-a20f-4bd5e8397718 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:42.377886 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea056f-29c0-4380-b9dc-2633f7a79e83 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:42.433167 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:42.484870 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:42.510585 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1cea2d-0a2b-4746-a20f-4bd5e8397718 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:42.518245 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea056f-29c0-4380-b9dc-2633f7a79e83 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:42.549869 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dadacbd-1e96-43a0-bbea-c257b077548a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:42.551417 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dadacbd-1e96-43a0-bbea-c257b077548a None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:42.552253 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dadacbd-1e96-43a0-bbea-c257b077548a None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:42.552532 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dadacbd-1e96-43a0-bbea-c257b077548a None admin] PATH_INFO: `/v3/users/9eb540242e364ea7955ea951382ed935` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:42.553847 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dadacbd-1e96-43a0-bbea-c257b077548a None admin] RBAC: Authorizing `identity:get_user(user_id=9eb540242e364ea7955ea951382ed935)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:42.555335 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dadacbd-1e96-43a0-bbea-c257b077548a None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:42.557315 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 75/149] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:39:42 2022] GET /identity/v3/users/9eb540242e364ea7955ea951382ed935 => generated 289 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:42.561153 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2932ea3-05fd-47cd-b170-e9e533fb3849 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:42.561448 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2932ea3-05fd-47cd-b170-e9e533fb3849 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:42.561749 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2932ea3-05fd-47cd-b170-e9e533fb3849 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:42.563036 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 76/150] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:42.566783 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2932ea3-05fd-47cd-b170-e9e533fb3849 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:42.573600 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fffcc3-294d-4733-b536-b3e23bf9ccf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:42.574294 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fffcc3-294d-4733-b536-b3e23bf9ccf8 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:42.574578 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fffcc3-294d-4733-b536-b3e23bf9ccf8 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:42.574859 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fffcc3-294d-4733-b536-b3e23bf9ccf8 None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:42.575627 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fffcc3-294d-4733-b536-b3e23bf9ccf8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:42.575938 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fffcc3-294d-4733-b536-b3e23bf9ccf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:42.577380 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fffcc3-294d-4733-b536-b3e23bf9ccf8 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:42.592882 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 75/151] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:39:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:42.594054 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367aae3e-8695-409e-82a8-8efc42f1cda4 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:42.594393 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367aae3e-8695-409e-82a8-8efc42f1cda4 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:42.594625 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367aae3e-8695-409e-82a8-8efc42f1cda4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:42.631065 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-367aae3e-8695-409e-82a8-8efc42f1cda4 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:42.631653 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-367aae3e-8695-409e-82a8-8efc42f1cda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:42.694644 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2932ea3-05fd-47cd-b170-e9e533fb3849 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:42.743279 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6431c7ab-83e6-4403-a10a-8a8f675d979b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:42.744173 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431c7ab-83e6-4403-a10a-8a8f675d979b None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:42.744456 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431c7ab-83e6-4403-a10a-8a8f675d979b None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:42.744733 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431c7ab-83e6-4403-a10a-8a8f675d979b None admin] PATH_INFO: `/v3/projects/fde6b97a4d09416a81257067f1d1b919` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:42.755323 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6431c7ab-83e6-4403-a10a-8a8f675d979b None admin] RBAC: Authorizing `identity:get_project(project_id=fde6b97a4d09416a81257067f1d1b919)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:42.756302 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 76/152] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:39:42.757398 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367aae3e-8695-409e-82a8-8efc42f1cda4 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:42.764104 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6431c7ab-83e6-4403-a10a-8a8f675d979b None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:42.766289 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 77/153] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:39:42 2022] GET /identity/v3/projects/fde6b97a4d09416a81257067f1d1b919 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:42.773720 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6431c7ab-83e6-4403-a10a-8a8f675d979b None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:42.813088 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:42.838071 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:42.934819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6431c7ab-83e6-4403-a10a-8a8f675d979b None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:42.982744 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9997d397-93d8-41ad-badf-f291c44fdd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:42.984055 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9997d397-93d8-41ad-badf-f291c44fdd28 None admin] REQUEST_METHOD: `PUT` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:42.985778 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9997d397-93d8-41ad-badf-f291c44fdd28 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:42.986406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9997d397-93d8-41ad-badf-f291c44fdd28 None admin] PATH_INFO: `/v3/projects/fde6b97a4d09416a81257067f1d1b919/users/9eb540242e364ea7955ea951382ed935/roles/e7249313e5334765ba16793745caed63` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:42.992625 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9997d397-93d8-41ad-badf-f291c44fdd28 None admin] RBAC: Authorizing `identity:create_grant(project_id=fde6b97a4d09416a81257067f1d1b919, user_id=9eb540242e364ea7955ea951382ed935, role_id=e7249313e5334765ba16793745caed63)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:42.995338 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9997d397-93d8-41ad-badf-f291c44fdd28 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:43.012968 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367aae3e-8695-409e-82a8-8efc42f1cda4 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:43.059854 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5c887-80a2-43a0-9fcd-bcc2ce558a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:43.061029 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5c887-80a2-43a0-9fcd-bcc2ce558a49 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:43.061679 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5c887-80a2-43a0-9fcd-bcc2ce558a49 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:43.062105 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5c887-80a2-43a0-9fcd-bcc2ce558a49 None admin] PATH_INFO: `/v3/users/45e7917424634d8897c1eb02702d4bc2` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:43.065744 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5c887-80a2-43a0-9fcd-bcc2ce558a49 None admin] RBAC: Authorizing `identity:get_user(user_id=45e7917424634d8897c1eb02702d4bc2)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:43.068420 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc5c887-80a2-43a0-9fcd-bcc2ce558a49 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:43.070776 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 77/154] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:39:42 2022] GET /identity/v3/users/45e7917424634d8897c1eb02702d4bc2 => generated 247 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:43.072123 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5c887-80a2-43a0-9fcd-bcc2ce558a49 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:43.093074 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 78/155] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 17 16:39:42 2022] PUT /identity/v3/projects/fde6b97a4d09416a81257067f1d1b919/users/9eb540242e364ea7955ea951382ed935/roles/e7249313e5334765ba16793745caed63 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 17 16:39:43.094365 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9997d397-93d8-41ad-badf-f291c44fdd28 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:43.255834 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:43.320987 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:43.331157 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9997d397-93d8-41ad-badf-f291c44fdd28 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:43.346977 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5c887-80a2-43a0-9fcd-bcc2ce558a49 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:43.409669 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c9b05e-dbfe-41d8-96cb-36039d451f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:43.410867 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9b05e-dbfe-41d8-96cb-36039d451f77 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:43.411168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9b05e-dbfe-41d8-96cb-36039d451f77 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:43.411491 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9b05e-dbfe-41d8-96cb-36039d451f77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:43.424901 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d64f0-19ed-480f-80e4-f0f1358d0412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:43.426793 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d64f0-19ed-480f-80e4-f0f1358d0412 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:43.427492 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d64f0-19ed-480f-80e4-f0f1358d0412 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:43.427984 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d64f0-19ed-480f-80e4-f0f1358d0412 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:43.448963 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963d64f0-19ed-480f-80e4-f0f1358d0412 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:39:43.458751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-963d64f0-19ed-480f-80e4-f0f1358d0412 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 17 16:39:43.458751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:39:43.458751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:39:43.458751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:39:43.458751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:39:43.458751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:39:43.458751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:39:43.458751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:39:43.458751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:39:43.458751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:39:43.458751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:39:43.458751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:39:43.458751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:39:43.458751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:39:43.458751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:39:43.458751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:39:43.459902 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:39:43.459902 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:39:43.459902 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:39:43.459902 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:39:43.459902 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:39:43.459902 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:39:43.459902 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:39:43.459902 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:39:43.459902 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:39:43.459902 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:39:43.459902 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:39:43.459902 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:39:43.459902 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:39:43.459902 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:39:43.459902 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:39:43.459902 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:39:43.462706 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:39:43.462706 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:39:43.462706 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:39:43.462706 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:39:43.462706 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:39:43.462706 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:39:43.462706 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:39:43.462706 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:39:43.462706 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:39:43.462706 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:39:43.462706 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 17 16:39:43.462706 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:39:43.464866 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 78/156] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 17 16:39:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:39:43.466523 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a60be7-b382-4942-b51b-933d735e8598 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:43.466961 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a60be7-b382-4942-b51b-933d735e8598 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:43.467375 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a60be7-b382-4942-b51b-933d735e8598 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:43.468956 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 79/157] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:43.475343 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a60be7-b382-4942-b51b-933d735e8598 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:43.512973 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c9b05e-dbfe-41d8-96cb-36039d451f77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:43.514533 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c9b05e-dbfe-41d8-96cb-36039d451f77 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:43.630693 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:43.670509 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 79/158] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:39:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 17 16:39:43.674852 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d58143-d528-40e8-a394-befb1de97d7a None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:43.674852 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d58143-d528-40e8-a394-befb1de97d7a None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:43.674852 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d58143-d528-40e8-a394-befb1de97d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:43.676553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a60be7-b382-4942-b51b-933d735e8598 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:43.705886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-d7d58143-d528-40e8-a394-befb1de97d7a None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:43.707269 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-d7d58143-d528-40e8-a394-befb1de97d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:43.750731 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd90f56-c0b0-4b36-a48b-9b005ad2459b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:43.751717 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd90f56-c0b0-4b36-a48b-9b005ad2459b None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:43.752218 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd90f56-c0b0-4b36-a48b-9b005ad2459b None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:43.752698 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd90f56-c0b0-4b36-a48b-9b005ad2459b None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:43.754182 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd90f56-c0b0-4b36-a48b-9b005ad2459b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:43.754815 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd90f56-c0b0-4b36-a48b-9b005ad2459b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:43.756635 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd90f56-c0b0-4b36-a48b-9b005ad2459b None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:43.773521 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 80/159] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:39:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:43.774761 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd90f56-c0b0-4b36-a48b-9b005ad2459b None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:43.804076 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:43.807560 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 80/160] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:39:43.808538 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d58143-d528-40e8-a394-befb1de97d7a None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:43.961556 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d58143-d528-40e8-a394-befb1de97d7a None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:43.964184 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:44.006066 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd90f56-c0b0-4b36-a48b-9b005ad2459b None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:44.008599 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51758d74-dc27-4d44-bcec-6c825baa63d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:44.009753 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51758d74-dc27-4d44-bcec-6c825baa63d6 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:44.010187 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51758d74-dc27-4d44-bcec-6c825baa63d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:44.010610 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51758d74-dc27-4d44-bcec-6c825baa63d6 None admin] PATH_INFO: `/v3/users/f6118de0629f4d08aa5388377f28b7ac` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:44.012000 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51758d74-dc27-4d44-bcec-6c825baa63d6 None admin] RBAC: Authorizing `identity:get_user(user_id=f6118de0629f4d08aa5388377f28b7ac)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:44.013632 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51758d74-dc27-4d44-bcec-6c825baa63d6 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:44.015461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 81/161] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:39:43 2022] GET /identity/v3/users/f6118de0629f4d08aa5388377f28b7ac => generated 275 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:44.016397 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51758d74-dc27-4d44-bcec-6c825baa63d6 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:44.101593 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ddd40-a43c-4e67-9719-c973587b304a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:44.102605 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ddd40-a43c-4e67-9719-c973587b304a None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:44.102785 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ddd40-a43c-4e67-9719-c973587b304a None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:44.103060 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ddd40-a43c-4e67-9719-c973587b304a None admin] PATH_INFO: `/v3/projects/fde6b97a4d09416a81257067f1d1b919` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:44.106686 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ddd40-a43c-4e67-9719-c973587b304a None admin] RBAC: Authorizing `identity:get_project(project_id=fde6b97a4d09416a81257067f1d1b919)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:44.108246 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ddd40-a43c-4e67-9719-c973587b304a None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:44.117086 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 81/162] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:39:43 2022] GET /identity/v3/projects/fde6b97a4d09416a81257067f1d1b919 => generated 298 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:44.117086 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ddd40-a43c-4e67-9719-c973587b304a None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:44.175890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51758d74-dc27-4d44-bcec-6c825baa63d6 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:44.226868 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98dcd7-916c-4339-8ead-6ca6f1399c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:44.227687 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98dcd7-916c-4339-8ead-6ca6f1399c74 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:44.227968 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98dcd7-916c-4339-8ead-6ca6f1399c74 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:44.228253 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98dcd7-916c-4339-8ead-6ca6f1399c74 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:44.241033 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa98dcd7-916c-4339-8ead-6ca6f1399c74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:39:44.253519 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-aa98dcd7-916c-4339-8ead-6ca6f1399c74 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:39:44.253519 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:39:44.253519 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:39:44.253519 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:39:44.253519 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:39:44.253519 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:39:44.253519 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:39:44.253519 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:39:44.253519 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:39:44.253519 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:39:44.253519 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:39:44.253519 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:39:44.253519 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:39:44.253519 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:39:44.253519 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:39:44.253519 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:39:44.254896 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:39:44.254896 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:39:44.254896 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:39:44.254896 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:39:44.254896 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:39:44.254896 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:39:44.254896 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:39:44.254896 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:39:44.254896 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:39:44.254896 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:39:44.254896 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:39:44.254896 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:39:44.254896 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:39:44.254896 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:39:44.254896 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:39:44.254896 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:39:44.256985 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:39:44.256985 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:39:44.256985 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:39:44.256985 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:39:44.256985 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:39:44.256985 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:39:44.256985 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:39:44.256985 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:39:44.256985 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:39:44.256985 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:39:44.256985 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:39:44.256985 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:39:44.256985 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 82/163] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:39:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:39:44.256985 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98dcd7-916c-4339-8ead-6ca6f1399c74 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:44.331072 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:44.331248 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:44.399186 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ddd40-a43c-4e67-9719-c973587b304a None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:44.424266 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98dcd7-916c-4339-8ead-6ca6f1399c74 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:44.464092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dd3315-46d7-4454-a8a4-9ccc90f9ef91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:44.464918 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd3315-46d7-4454-a8a4-9ccc90f9ef91 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:44.465219 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd3315-46d7-4454-a8a4-9ccc90f9ef91 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:44.465504 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd3315-46d7-4454-a8a4-9ccc90f9ef91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:44.466373 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dd3315-46d7-4454-a8a4-9ccc90f9ef91 None admin] RBAC: Adding query filter params (role.id=1025051975f64f689effa58dc4259e36, scope.project.id=fde6b97a4d09416a81257067f1d1b919, user.id=45e7917424634d8897c1eb02702d4bc2) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:44.466694 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dd3315-46d7-4454-a8a4-9ccc90f9ef91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:44.468171 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dd3315-46d7-4454-a8a4-9ccc90f9ef91 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:44.475063 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 83/164] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:39:44 2022] GET /identity/v3/role_assignments?user.id=45e7917424634d8897c1eb02702d4bc2&scope.project.id=fde6b97a4d09416a81257067f1d1b919&role.id=1025051975f64f689effa58dc4259e36 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:44.475905 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dd3315-46d7-4454-a8a4-9ccc90f9ef91 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:44.514115 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c985ba-ded7-4edd-833e-18572734a73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:44.514804 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c985ba-ded7-4edd-833e-18572734a73e None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:44.515086 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c985ba-ded7-4edd-833e-18572734a73e None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:44.515361 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c985ba-ded7-4edd-833e-18572734a73e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:44.622215 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:44.631073 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dd3315-46d7-4454-a8a4-9ccc90f9ef91 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:44.669016 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c985ba-ded7-4edd-833e-18572734a73e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:44.670749 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c985ba-ded7-4edd-833e-18572734a73e None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:44.677834 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4766e-7475-4271-8f7c-c2597d7ba4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:44.678958 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4766e-7475-4271-8f7c-c2597d7ba4db None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:44.679473 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4766e-7475-4271-8f7c-c2597d7ba4db None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:44.679847 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4766e-7475-4271-8f7c-c2597d7ba4db None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:44.680922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4766e-7475-4271-8f7c-c2597d7ba4db None admin] RBAC: Adding query filter params (name=member) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:44.681533 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4766e-7475-4271-8f7c-c2597d7ba4db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:44.683436 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4766e-7475-4271-8f7c-c2597d7ba4db None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:44.698599 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 84/165] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 17 16:39:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:44.705410 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4766e-7475-4271-8f7c-c2597d7ba4db None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:44.829194 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4766e-7475-4271-8f7c-c2597d7ba4db None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:44.868240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688554bd-1029-40d9-aa65-96adc4db9cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:44.869328 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688554bd-1029-40d9-aa65-96adc4db9cec None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:44.869622 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688554bd-1029-40d9-aa65-96adc4db9cec None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:44.869909 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688554bd-1029-40d9-aa65-96adc4db9cec None admin] PATH_INFO: `/v3/users/f6118de0629f4d08aa5388377f28b7ac` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:44.871277 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688554bd-1029-40d9-aa65-96adc4db9cec None admin] RBAC: Authorizing `identity:get_user(user_id=f6118de0629f4d08aa5388377f28b7ac)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:44.872806 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688554bd-1029-40d9-aa65-96adc4db9cec None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:44.874587 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 85/166] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:39:44 2022] GET /identity/v3/users/f6118de0629f4d08aa5388377f28b7ac => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:44.880850 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688554bd-1029-40d9-aa65-96adc4db9cec None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:44.917237 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:44.934881 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:45.002836 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688554bd-1029-40d9-aa65-96adc4db9cec None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:45.009158 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 82/167] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:39:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 894 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 17 16:39:45.015381 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c985ba-ded7-4edd-833e-18572734a73e None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:45.043380 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc6eb2-6be7-4d3c-a4e1-972930860cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:45.044059 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc6eb2-6be7-4d3c-a4e1-972930860cc8 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:45.044349 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc6eb2-6be7-4d3c-a4e1-972930860cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:45.044633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc6eb2-6be7-4d3c-a4e1-972930860cc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:45.123101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bc6eb2-6be7-4d3c-a4e1-972930860cc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:45.124651 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bc6eb2-6be7-4d3c-a4e1-972930860cc8 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:45.157181 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:45.240884 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 86/168] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:39:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 17 16:39:45.246717 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc6eb2-6be7-4d3c-a4e1-972930860cc8 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:45.364027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc6eb2-6be7-4d3c-a4e1-972930860cc8 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:45.399631 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:45.407884 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e3733-d1b1-45cc-94e5-0c7390cb6eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:45.409059 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e3733-d1b1-45cc-94e5-0c7390cb6eb2 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:45.409350 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e3733-d1b1-45cc-94e5-0c7390cb6eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:45.410215 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e3733-d1b1-45cc-94e5-0c7390cb6eb2 None admin] PATH_INFO: `/v3/projects/fde6b97a4d09416a81257067f1d1b919` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:45.412012 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687e3733-d1b1-45cc-94e5-0c7390cb6eb2 None admin] RBAC: Authorizing `identity:get_project(project_id=fde6b97a4d09416a81257067f1d1b919)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:45.413509 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687e3733-d1b1-45cc-94e5-0c7390cb6eb2 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:45.415629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 87/169] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:39:45 2022] GET /identity/v3/projects/fde6b97a4d09416a81257067f1d1b919 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:45.422168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e3733-d1b1-45cc-94e5-0c7390cb6eb2 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:45.445130 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c985ba-ded7-4edd-833e-18572734a73e None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:45.564251 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e3733-d1b1-45cc-94e5-0c7390cb6eb2 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:45.617568 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9915e6a-048e-4af3-b56a-806a2d4ed533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:45.618457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9915e6a-048e-4af3-b56a-806a2d4ed533 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:45.618745 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9915e6a-048e-4af3-b56a-806a2d4ed533 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:45.619029 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9915e6a-048e-4af3-b56a-806a2d4ed533 None admin] PATH_INFO: `/v3/projects/fde6b97a4d09416a81257067f1d1b919` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:45.625615 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9915e6a-048e-4af3-b56a-806a2d4ed533 None admin] RBAC: Authorizing `identity:get_project(project_id=fde6b97a4d09416a81257067f1d1b919)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:45.627063 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c45a01-f092-4d20-9b1c-d4c0f6b0d00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:45.627186 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9915e6a-048e-4af3-b56a-806a2d4ed533 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:45.628189 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c45a01-f092-4d20-9b1c-d4c0f6b0d00e None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:45.628734 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c45a01-f092-4d20-9b1c-d4c0f6b0d00e None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:45.629263 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c45a01-f092-4d20-9b1c-d4c0f6b0d00e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:45.632680 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 83/170] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:39:45 2022] GET /identity/v3/projects/fde6b97a4d09416a81257067f1d1b919 => generated 298 bytes in 615 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:45.632814 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c45a01-f092-4d20-9b1c-d4c0f6b0d00e None admin] RBAC: Adding query filter params (role.id=a47f5df8505f49419d0acb4f0f287610, scope.project.id=fde6b97a4d09416a81257067f1d1b919, user.id=f6118de0629f4d08aa5388377f28b7ac) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:45.632814 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c45a01-f092-4d20-9b1c-d4c0f6b0d00e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:45.634163 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c45a01-f092-4d20-9b1c-d4c0f6b0d00e None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:45.635111 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9915e6a-048e-4af3-b56a-806a2d4ed533 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:45.658994 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 88/171] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:39:45 2022] GET /identity/v3/role_assignments?user.id=f6118de0629f4d08aa5388377f28b7ac&scope.project.id=fde6b97a4d09416a81257067f1d1b919&role.id=a47f5df8505f49419d0acb4f0f287610 => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:45.668203 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67833819-da0d-44d4-b278-de9f10681264 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:45.668650 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67833819-da0d-44d4-b278-de9f10681264 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:45.669124 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67833819-da0d-44d4-b278-de9f10681264 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:45.671393 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 89/172] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:39:45.680394 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82a136-8704-4ee9-9ab6-8707e6b02c21 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:45.680927 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82a136-8704-4ee9-9ab6-8707e6b02c21 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:45.681433 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82a136-8704-4ee9-9ab6-8707e6b02c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:45.727017 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:45.730570 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-eb82a136-8704-4ee9-9ab6-8707e6b02c21 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:45.732415 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-eb82a136-8704-4ee9-9ab6-8707e6b02c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:45.810689 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9915e6a-048e-4af3-b56a-806a2d4ed533 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:45.859475 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb116d2-5cd5-4e5a-af20-f01afe617ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:45.860545 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb116d2-5cd5-4e5a-af20-f01afe617ecc None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:45.860874 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb116d2-5cd5-4e5a-af20-f01afe617ecc None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:45.861165 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb116d2-5cd5-4e5a-af20-f01afe617ecc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:45.862047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb116d2-5cd5-4e5a-af20-f01afe617ecc None admin] RBAC: Adding query filter params (role.id=71cf67eb04704663aff719f6e3804f1a, scope.project.id=fde6b97a4d09416a81257067f1d1b919, user.id=f6118de0629f4d08aa5388377f28b7ac) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:45.862370 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb116d2-5cd5-4e5a-af20-f01afe617ecc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:45.864817 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb116d2-5cd5-4e5a-af20-f01afe617ecc None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:45.872114 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 84/173] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:39:45 2022] GET /identity/v3/role_assignments?user.id=f6118de0629f4d08aa5388377f28b7ac&scope.project.id=fde6b97a4d09416a81257067f1d1b919&role.id=71cf67eb04704663aff719f6e3804f1a => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:45.873608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95acd2a-7e99-4b6d-8137-c8386b5759b4 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:45.873908 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95acd2a-7e99-4b6d-8137-c8386b5759b4 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:45.874196 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95acd2a-7e99-4b6d-8137-c8386b5759b4 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:45.875601 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 85/174] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:45.890424 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba084e5-c825-4652-a7bf-d1b4cfdfb7ca None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:45.890731 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba084e5-c825-4652-a7bf-d1b4cfdfb7ca None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:45.891006 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba084e5-c825-4652-a7bf-d1b4cfdfb7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:45.919912 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-bba084e5-c825-4652-a7bf-d1b4cfdfb7ca None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:45.921269 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-bba084e5-c825-4652-a7bf-d1b4cfdfb7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:45.970710 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:46.048099 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 86/175] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:39:46.050613 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 90/176] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:39:46.118036 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba084e5-c825-4652-a7bf-d1b4cfdfb7ca None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:46.121690 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb82a136-8704-4ee9-9ab6-8707e6b02c21 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:46.193172 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:46.255322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba084e5-c825-4652-a7bf-d1b4cfdfb7ca None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:46.303095 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0323bf3-55b4-45f6-954e-c54b3cdd50a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:46.303916 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0323bf3-55b4-45f6-954e-c54b3cdd50a2 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:46.304207 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0323bf3-55b4-45f6-954e-c54b3cdd50a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:46.304490 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0323bf3-55b4-45f6-954e-c54b3cdd50a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:46.331040 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0323bf3-55b4-45f6-954e-c54b3cdd50a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:39:46.341242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-c0323bf3-55b4-45f6-954e-c54b3cdd50a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:39:46.341242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:39:46.341242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:39:46.341242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:39:46.341242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:39:46.341242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:39:46.341242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:39:46.341242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:39:46.341242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:39:46.341242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:39:46.341242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:39:46.341242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:39:46.341242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:39:46.341242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:39:46.341242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:39:46.341242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:39:46.342173 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:39:46.342173 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:39:46.342173 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:39:46.342173 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:39:46.342173 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:39:46.342173 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:39:46.342173 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:39:46.342173 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:39:46.342173 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:39:46.342173 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:39:46.342173 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:39:46.342173 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:39:46.342173 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:39:46.342173 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:39:46.342173 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:39:46.342173 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:39:46.342956 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:39:46.342956 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:39:46.342956 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:39:46.342956 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:39:46.342956 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:39:46.342956 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:39:46.342956 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:39:46.342956 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:39:46.342956 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:39:46.342956 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:39:46.342956 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:39:46.342956 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:39:46.346144 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 87/177] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:39:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:39:46.356552 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0323bf3-55b4-45f6-954e-c54b3cdd50a2 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:46.395914 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb82a136-8704-4ee9-9ab6-8707e6b02c21 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:46.475299 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57813774-5b11-4558-99fd-71bdab687b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:46.476671 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57813774-5b11-4558-99fd-71bdab687b3f None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:46.477271 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57813774-5b11-4558-99fd-71bdab687b3f None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:46.477747 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57813774-5b11-4558-99fd-71bdab687b3f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:46.499845 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57813774-5b11-4558-99fd-71bdab687b3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:39:46.505847 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0323bf3-55b4-45f6-954e-c54b3cdd50a2 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:46.508368 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:46.515166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-57813774-5b11-4558-99fd-71bdab687b3f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:39:46.515166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:39:46.515166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:39:46.515166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:39:46.515166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:39:46.515166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:39:46.515166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:39:46.515166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:39:46.515166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:39:46.515166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:39:46.515166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:39:46.515166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:39:46.515166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:39:46.515166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:39:46.515166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:39:46.515166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:39:46.516063 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:39:46.516063 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:39:46.516063 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:39:46.516063 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:39:46.516063 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:39:46.516063 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:39:46.516063 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:39:46.516063 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:39:46.516063 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:39:46.516063 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:39:46.516063 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:39:46.516063 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:39:46.516063 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:39:46.516063 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:39:46.516063 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:39:46.516063 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:39:46.516861 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:39:46.516861 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:39:46.516861 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:39:46.516861 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:39:46.516861 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:39:46.516861 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:39:46.516861 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:39:46.516861 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:39:46.516861 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:39:46.516861 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:39:46.516861 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:39:46.516861 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:39:46.517940 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 91/178] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:39:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:39:46.520354 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8db475-64f8-4fe7-a1ae-670b7620240a None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:46.520884 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8db475-64f8-4fe7-a1ae-670b7620240a None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:46.521400 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8db475-64f8-4fe7-a1ae-670b7620240a None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:46.524338 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 92/179] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:46.529892 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8db475-64f8-4fe7-a1ae-670b7620240a None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:46.554983 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d18cf-9b5f-4e4b-a44d-7945400ca687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:46.555677 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d18cf-9b5f-4e4b-a44d-7945400ca687 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:46.555963 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d18cf-9b5f-4e4b-a44d-7945400ca687 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:46.556239 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d18cf-9b5f-4e4b-a44d-7945400ca687 None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:46.557042 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893d18cf-9b5f-4e4b-a44d-7945400ca687 None admin] RBAC: Adding query filter params (name=member) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:46.557357 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893d18cf-9b5f-4e4b-a44d-7945400ca687 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:46.558772 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893d18cf-9b5f-4e4b-a44d-7945400ca687 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:46.570785 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 88/180] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 17 16:39:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:46.572375 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f067765-2ab9-4618-9667-0299c4feafeb None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:46.574296 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f067765-2ab9-4618-9667-0299c4feafeb None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:46.574590 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f067765-2ab9-4618-9667-0299c4feafeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:46.596453 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-4f067765-2ab9-4618-9667-0299c4feafeb None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:46.598556 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-4f067765-2ab9-4618-9667-0299c4feafeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:46.617425 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:46.681095 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 89/181] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:39:46.682596 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b7f52-2c03-4736-9e78-e410a8c93192 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:46.682895 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b7f52-2c03-4736-9e78-e410a8c93192 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:46.683178 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b7f52-2c03-4736-9e78-e410a8c93192 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:46.689350 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 90/182] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:46.689350 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b7f52-2c03-4736-9e78-e410a8c93192 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:46.694303 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8db475-64f8-4fe7-a1ae-670b7620240a None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:46.736852 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be9db1-9c52-4f92-b994-d348608d61dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:46.737547 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be9db1-9c52-4f92-b994-d348608d61dd None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:46.738493 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be9db1-9c52-4f92-b994-d348608d61dd None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:46.739389 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be9db1-9c52-4f92-b994-d348608d61dd None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:46.740174 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41be9db1-9c52-4f92-b994-d348608d61dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:46.740492 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41be9db1-9c52-4f92-b994-d348608d61dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:46.742721 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41be9db1-9c52-4f92-b994-d348608d61dd None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:46.752142 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 93/183] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:39:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:46.754048 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae89610-c2d3-4b17-8d15-31bdbc2f28ec None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:46.754343 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae89610-c2d3-4b17-8d15-31bdbc2f28ec None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:46.754627 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae89610-c2d3-4b17-8d15-31bdbc2f28ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:46.772281 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-fae89610-c2d3-4b17-8d15-31bdbc2f28ec None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:46.773533 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-fae89610-c2d3-4b17-8d15-31bdbc2f28ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:46.809125 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:46.817253 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b7f52-2c03-4736-9e78-e410a8c93192 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:46.854438 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 94/184] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:39:46.856398 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb702d3-b7fa-4dca-8e44-8ed4eb81efda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:46.857346 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb702d3-b7fa-4dca-8e44-8ed4eb81efda None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:46.857638 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb702d3-b7fa-4dca-8e44-8ed4eb81efda None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:46.857918 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb702d3-b7fa-4dca-8e44-8ed4eb81efda None admin] PATH_INFO: `/v3/users/f6118de0629f4d08aa5388377f28b7ac` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:46.859169 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb702d3-b7fa-4dca-8e44-8ed4eb81efda None admin] RBAC: Authorizing `identity:get_user(user_id=f6118de0629f4d08aa5388377f28b7ac)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:46.860685 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb702d3-b7fa-4dca-8e44-8ed4eb81efda None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:46.862659 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 91/185] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:39:46 2022] GET /identity/v3/users/f6118de0629f4d08aa5388377f28b7ac => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:46.863848 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb702d3-b7fa-4dca-8e44-8ed4eb81efda None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:46.864203 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae89610-c2d3-4b17-8d15-31bdbc2f28ec None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:46.923791 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:46.990405 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae89610-c2d3-4b17-8d15-31bdbc2f28ec None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:46.996280 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb702d3-b7fa-4dca-8e44-8ed4eb81efda None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:47.031523 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc3e0c-7949-45eb-a1dd-61036d74f44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:47.032371 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc3e0c-7949-45eb-a1dd-61036d74f44f None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:47.032656 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc3e0c-7949-45eb-a1dd-61036d74f44f None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:47.032980 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc3e0c-7949-45eb-a1dd-61036d74f44f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:47.042068 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1b2c7-8ef9-4f87-ac30-3f94c0387088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:47.042954 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1b2c7-8ef9-4f87-ac30-3f94c0387088 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:47.043242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1b2c7-8ef9-4f87-ac30-3f94c0387088 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:47.043533 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1b2c7-8ef9-4f87-ac30-3f94c0387088 None admin] PATH_INFO: `/v3/users/9337ba85d24f434fabd701c84d5cd89a` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:47.044642 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cc3e0c-7949-45eb-a1dd-61036d74f44f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:39:47.053889 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-06cc3e0c-7949-45eb-a1dd-61036d74f44f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 17 16:39:47.053889 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:39:47.053889 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:39:47.053889 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:39:47.053889 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:39:47.053889 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:39:47.053889 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:39:47.053889 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:39:47.053889 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:39:47.053889 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:39:47.053889 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:39:47.053889 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:39:47.053889 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:39:47.053889 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:39:47.053889 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:39:47.053889 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:39:47.055273 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:39:47.055273 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:39:47.055273 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:39:47.055273 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:39:47.055273 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:39:47.055273 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:39:47.055273 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:39:47.055273 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:39:47.055273 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:39:47.055273 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:39:47.055273 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:39:47.055273 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:39:47.055273 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:39:47.055273 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:39:47.055273 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:39:47.055273 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:39:47.056665 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:39:47.056665 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:39:47.056665 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:39:47.056665 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:39:47.056665 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:39:47.056665 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:39:47.056665 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:39:47.056665 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:39:47.056665 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:39:47.056665 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:39:47.056665 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 17 16:39:47.056665 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:39:47.056665 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 95/186] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 17 16:39:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:39:47.058848 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc3e0c-7949-45eb-a1dd-61036d74f44f None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:47.060037 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf1b2c7-8ef9-4f87-ac30-3f94c0387088 None admin] RBAC: Authorizing `identity:get_user(user_id=9337ba85d24f434fabd701c84d5cd89a)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:47.061651 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf1b2c7-8ef9-4f87-ac30-3f94c0387088 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:47.063399 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 92/187] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:39:46 2022] GET /identity/v3/users/9337ba85d24f434fabd701c84d5cd89a => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:47.064172 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1b2c7-8ef9-4f87-ac30-3f94c0387088 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:47.090039 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:47.160026 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:47.163301 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc3e0c-7949-45eb-a1dd-61036d74f44f None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:47.167190 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1b2c7-8ef9-4f87-ac30-3f94c0387088 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:47.199752 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e544d-14c3-49f3-a051-28b0443d59dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:47.200420 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e544d-14c3-49f3-a051-28b0443d59dd None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:47.200706 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e544d-14c3-49f3-a051-28b0443d59dd None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:47.201016 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e544d-14c3-49f3-a051-28b0443d59dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:47.203214 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dbbe36-9c15-4e88-a201-35072215eeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:47.204018 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbbe36-9c15-4e88-a201-35072215eeca None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:47.204299 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbbe36-9c15-4e88-a201-35072215eeca None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:47.204581 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbbe36-9c15-4e88-a201-35072215eeca None admin] PATH_INFO: `/v3/domains/default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:47.205829 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dbbe36-9c15-4e88-a201-35072215eeca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:47.208289 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dbbe36-9c15-4e88-a201-35072215eeca None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:47.209974 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 93/188] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:39:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:47.211202 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5eb7c-e240-4dc9-aebd-60a7d370902d None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:47.211488 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5eb7c-e240-4dc9-aebd-60a7d370902d None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:47.211766 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5eb7c-e240-4dc9-aebd-60a7d370902d None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:47.213040 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 94/189] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:47.213753 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5eb7c-e240-4dc9-aebd-60a7d370902d None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:47.277492 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034e544d-14c3-49f3-a051-28b0443d59dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:47.278963 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034e544d-14c3-49f3-a051-28b0443d59dd None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:47.311549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:47.333213 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5eb7c-e240-4dc9-aebd-60a7d370902d None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:47.390598 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbcc3bc-ecd6-498d-9721-3a56ef2e96ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:47.391721 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbcc3bc-ecd6-498d-9721-3a56ef2e96ae None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:47.392201 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbcc3bc-ecd6-498d-9721-3a56ef2e96ae None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:47.392641 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbcc3bc-ecd6-498d-9721-3a56ef2e96ae None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:47.393982 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbcc3bc-ecd6-498d-9721-3a56ef2e96ae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:47.394493 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbcc3bc-ecd6-498d-9721-3a56ef2e96ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:47.398577 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbcc3bc-ecd6-498d-9721-3a56ef2e96ae None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:47.417072 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 95/190] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:39:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:47.418913 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbcc3bc-ecd6-498d-9721-3a56ef2e96ae None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:47.442009 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:47.447446 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 96/191] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:39:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 17 16:39:47.448299 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e544d-14c3-49f3-a051-28b0443d59dd None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:47.561119 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbcc3bc-ecd6-498d-9721-3a56ef2e96ae None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:47.588823 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e544d-14c3-49f3-a051-28b0443d59dd None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:47.609683 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724fd4c7-385d-4349-a2d0-c954523ea9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:47.610353 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fd4c7-385d-4349-a2d0-c954523ea9aa None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:47.610647 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fd4c7-385d-4349-a2d0-c954523ea9aa None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:47.610920 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fd4c7-385d-4349-a2d0-c954523ea9aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:47.638731 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:47.639587 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfddb83-1e42-4c87-a7be-09bdfe968d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:47.640502 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfddb83-1e42-4c87-a7be-09bdfe968d1b None admin] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:47.640612 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfddb83-1e42-4c87-a7be-09bdfe968d1b None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:47.640873 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfddb83-1e42-4c87-a7be-09bdfe968d1b None admin] PATH_INFO: `/v3/users` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:47.641562 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfddb83-1e42-4c87-a7be-09bdfe968d1b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:47.643063 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfddb83-1e42-4c87-a7be-09bdfe968d1b None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:47.689494 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724fd4c7-385d-4349-a2d0-c954523ea9aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:47.691016 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724fd4c7-385d-4349-a2d0-c954523ea9aa None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:47.759462 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 97/192] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 17 16:39:47 2022] POST /identity/v3/users => generated 297 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:39:47.760910 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d7a95-8536-4725-a636-9e6a8f3a4a87 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:47.761277 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d7a95-8536-4725-a636-9e6a8f3a4a87 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:47.761663 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d7a95-8536-4725-a636-9e6a8f3a4a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:47.780841 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-fc2d7a95-8536-4725-a636-9e6a8f3a4a87 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:47.782137 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-fc2d7a95-8536-4725-a636-9e6a8f3a4a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:47.810921 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 96/193] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:39:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 17 16:39:47.810921 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724fd4c7-385d-4349-a2d0-c954523ea9aa None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:47.850126 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:47.869023 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 98/194] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:39:47.874540 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:47.881838 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d7a95-8536-4725-a636-9e6a8f3a4a87 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:47.935689 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724fd4c7-385d-4349-a2d0-c954523ea9aa None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:47.983458 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d773b395-1de8-4b9e-abb4-ad28406d76c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:47.984901 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773b395-1de8-4b9e-abb4-ad28406d76c9 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:47.985167 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773b395-1de8-4b9e-abb4-ad28406d76c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:47.985448 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d773b395-1de8-4b9e-abb4-ad28406d76c9 None admin] PATH_INFO: `/v3/users/9337ba85d24f434fabd701c84d5cd89a` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:47.987034 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d773b395-1de8-4b9e-abb4-ad28406d76c9 None admin] RBAC: Authorizing `identity:get_user(user_id=9337ba85d24f434fabd701c84d5cd89a)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:47.989647 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d773b395-1de8-4b9e-abb4-ad28406d76c9 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:47.991345 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 97/195] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:39:47 2022] GET /identity/v3/users/9337ba85d24f434fabd701c84d5cd89a => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:47.992125 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d773b395-1de8-4b9e-abb4-ad28406d76c9 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:48.015300 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d7a95-8536-4725-a636-9e6a8f3a4a87 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:48.060710 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb90f6-21b9-40f3-9443-d30c90c5331c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:48.061617 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb90f6-21b9-40f3-9443-d30c90c5331c None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:48.061904 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb90f6-21b9-40f3-9443-d30c90c5331c None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:48.062183 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb90f6-21b9-40f3-9443-d30c90c5331c None admin] PATH_INFO: `/v3/projects/1732be3d65f741438a4c11098484f9e0` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:48.063425 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bb90f6-21b9-40f3-9443-d30c90c5331c None admin] RBAC: Authorizing `identity:get_project(project_id=1732be3d65f741438a4c11098484f9e0)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:48.064929 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bb90f6-21b9-40f3-9443-d30c90c5331c None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:48.067654 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 99/196] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:39:47 2022] GET /identity/v3/projects/1732be3d65f741438a4c11098484f9e0 => generated 312 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:48.068452 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb90f6-21b9-40f3-9443-d30c90c5331c None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:48.133029 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d773b395-1de8-4b9e-abb4-ad28406d76c9 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:48.141471 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:48.161713 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:48.176615 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba24b65-4127-4617-a4e2-ee5b6fa02175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:48.178131 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba24b65-4127-4617-a4e2-ee5b6fa02175 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:48.178658 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba24b65-4127-4617-a4e2-ee5b6fa02175 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:48.179134 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba24b65-4127-4617-a4e2-ee5b6fa02175 None admin] PATH_INFO: `/v3/projects/6704fc6d36e84a078282ac2057784505` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:48.180632 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba24b65-4127-4617-a4e2-ee5b6fa02175 None admin] RBAC: Authorizing `identity:get_project(project_id=6704fc6d36e84a078282ac2057784505)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:48.182401 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba24b65-4127-4617-a4e2-ee5b6fa02175 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:48.185773 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 98/197] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:39:47 2022] GET /identity/v3/projects/6704fc6d36e84a078282ac2057784505 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:48.186871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba24b65-4127-4617-a4e2-ee5b6fa02175 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:48.201145 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb90f6-21b9-40f3-9443-d30c90c5331c None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:48.243804 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca624e-6f4a-4248-b49b-f44d36b86c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:48.244930 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca624e-6f4a-4248-b49b-f44d36b86c5e None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:48.245040 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca624e-6f4a-4248-b49b-f44d36b86c5e None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:48.245307 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca624e-6f4a-4248-b49b-f44d36b86c5e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:48.258248 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eca624e-6f4a-4248-b49b-f44d36b86c5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:39:48.266559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-4eca624e-6f4a-4248-b49b-f44d36b86c5e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 17 16:39:48.266559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:39:48.266559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:39:48.266559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:39:48.266559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:39:48.266559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:39:48.266559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:39:48.266559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:39:48.266559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:39:48.266559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:39:48.266559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:39:48.266559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:39:48.266559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:39:48.266559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:39:48.266559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:39:48.266559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:39:48.267450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:39:48.267450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:39:48.267450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:39:48.267450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:39:48.267450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:39:48.267450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:39:48.267450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:39:48.267450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:39:48.267450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:39:48.267450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:39:48.267450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:39:48.267450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:39:48.267450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:39:48.267450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:39:48.267450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:39:48.267450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:39:48.268274 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:39:48.268274 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:39:48.268274 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:39:48.268274 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:39:48.268274 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:39:48.268274 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:39:48.268274 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:39:48.268274 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:39:48.268274 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:39:48.268274 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:39:48.268274 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 17 16:39:48.268274 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:39:48.268274 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 100/198] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:39:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:39:48.269650 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca624e-6f4a-4248-b49b-f44d36b86c5e None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:48.316814 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba24b65-4127-4617-a4e2-ee5b6fa02175 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:48.359539 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d013564-4eb6-4d38-a1dc-9960bb9bfcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:48.360682 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d013564-4eb6-4d38-a1dc-9960bb9bfcb6 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:48.361214 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d013564-4eb6-4d38-a1dc-9960bb9bfcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:48.361716 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d013564-4eb6-4d38-a1dc-9960bb9bfcb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:48.387042 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca624e-6f4a-4248-b49b-f44d36b86c5e None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:48.403371 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:48.428145 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678cb2ea-09b7-4a22-a62f-dd67630de8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:48.428841 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678cb2ea-09b7-4a22-a62f-dd67630de8dd None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:48.429133 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678cb2ea-09b7-4a22-a62f-dd67630de8dd None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:48.429411 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678cb2ea-09b7-4a22-a62f-dd67630de8dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:48.430259 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678cb2ea-09b7-4a22-a62f-dd67630de8dd None admin] RBAC: Adding query filter params (role.id=a47f5df8505f49419d0acb4f0f287610, scope.project.id=1732be3d65f741438a4c11098484f9e0, user.id=f6118de0629f4d08aa5388377f28b7ac) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:48.430574 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678cb2ea-09b7-4a22-a62f-dd67630de8dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:48.432187 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678cb2ea-09b7-4a22-a62f-dd67630de8dd None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:48.432733 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:48.445186 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 101/199] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:39:48 2022] GET /identity/v3/role_assignments?user.id=f6118de0629f4d08aa5388377f28b7ac&scope.project.id=1732be3d65f741438a4c11098484f9e0&role.id=a47f5df8505f49419d0acb4f0f287610 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:48.446705 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678cb2ea-09b7-4a22-a62f-dd67630de8dd None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:48.449828 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d013564-4eb6-4d38-a1dc-9960bb9bfcb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:48.451492 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d013564-4eb6-4d38-a1dc-9960bb9bfcb6 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:48.591270 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 99/200] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:39:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 17 16:39:48.593429 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d013564-4eb6-4d38-a1dc-9960bb9bfcb6 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:48.667414 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678cb2ea-09b7-4a22-a62f-dd67630de8dd None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:48.725292 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:48.729173 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88490e65-ca8c-4996-8f1f-fded21b3b9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:48.730115 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88490e65-ca8c-4996-8f1f-fded21b3b9c2 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:48.730600 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88490e65-ca8c-4996-8f1f-fded21b3b9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:48.731089 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88490e65-ca8c-4996-8f1f-fded21b3b9c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:48.732195 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88490e65-ca8c-4996-8f1f-fded21b3b9c2 None admin] RBAC: Adding query filter params (role.id=1025051975f64f689effa58dc4259e36, scope.project.id=6704fc6d36e84a078282ac2057784505, user.id=9337ba85d24f434fabd701c84d5cd89a) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:48.732703 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88490e65-ca8c-4996-8f1f-fded21b3b9c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:48.734512 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88490e65-ca8c-4996-8f1f-fded21b3b9c2 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:48.740548 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d013564-4eb6-4d38-a1dc-9960bb9bfcb6 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:48.742682 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 102/201] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:39:48 2022] GET /identity/v3/role_assignments?user.id=9337ba85d24f434fabd701c84d5cd89a&scope.project.id=6704fc6d36e84a078282ac2057784505&role.id=1025051975f64f689effa58dc4259e36 => generated 259 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:48.743365 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88490e65-ca8c-4996-8f1f-fded21b3b9c2 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:48.781218 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d2573-f924-45b9-b5a3-16395bbd3c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:48.782005 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d2573-f924-45b9-b5a3-16395bbd3c0a None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:48.782301 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d2573-f924-45b9-b5a3-16395bbd3c0a None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:48.782586 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d2573-f924-45b9-b5a3-16395bbd3c0a None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:48.783462 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d2573-f924-45b9-b5a3-16395bbd3c0a None admin] RBAC: Adding query filter params (name=reader) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:48.783782 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d2573-f924-45b9-b5a3-16395bbd3c0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:48.786331 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508d2573-f924-45b9-b5a3-16395bbd3c0a None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:48.796860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 100/202] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 17 16:39:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:48.802964 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:48.803670 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d2573-f924-45b9-b5a3-16395bbd3c0a None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:48.869907 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88490e65-ca8c-4996-8f1f-fded21b3b9c2 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:48.913831 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9bccda-ad15-41ea-9824-54204d0079ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:48.915216 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bccda-ad15-41ea-9824-54204d0079ae None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:48.915681 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bccda-ad15-41ea-9824-54204d0079ae None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:48.916164 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bccda-ad15-41ea-9824-54204d0079ae None admin] PATH_INFO: `/v3/projects/6704fc6d36e84a078282ac2057784505` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:48.917925 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9bccda-ad15-41ea-9824-54204d0079ae None admin] RBAC: Authorizing `identity:get_project(project_id=6704fc6d36e84a078282ac2057784505)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:48.919862 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9bccda-ad15-41ea-9824-54204d0079ae None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:48.921802 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d2573-f924-45b9-b5a3-16395bbd3c0a None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:48.922360 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 103/203] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:39:48 2022] GET /identity/v3/projects/6704fc6d36e84a078282ac2057784505 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:48.931969 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9bccda-ad15-41ea-9824-54204d0079ae None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:48.976011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729886b7-4661-4b7f-960e-f9d556ed5bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:48.977446 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729886b7-4661-4b7f-960e-f9d556ed5bb5 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:48.977927 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729886b7-4661-4b7f-960e-f9d556ed5bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:48.978418 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729886b7-4661-4b7f-960e-f9d556ed5bb5 None admin] PATH_INFO: `/v3/users/9eb540242e364ea7955ea951382ed935` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:48.979967 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729886b7-4661-4b7f-960e-f9d556ed5bb5 None admin] RBAC: Authorizing `identity:get_user(user_id=9eb540242e364ea7955ea951382ed935)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:48.981754 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729886b7-4661-4b7f-960e-f9d556ed5bb5 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:48.984016 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 101/204] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:39:48 2022] GET /identity/v3/users/9eb540242e364ea7955ea951382ed935 => generated 289 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:48.990878 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729886b7-4661-4b7f-960e-f9d556ed5bb5 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:49.015922 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:49.099901 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9bccda-ad15-41ea-9824-54204d0079ae None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:49.110907 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:49.117097 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729886b7-4661-4b7f-960e-f9d556ed5bb5 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:49.139636 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31827dc1-eecb-41de-9708-e9ce76204eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:49.140305 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31827dc1-eecb-41de-9708-e9ce76204eb0 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:49.140650 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31827dc1-eecb-41de-9708-e9ce76204eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:49.140964 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31827dc1-eecb-41de-9708-e9ce76204eb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:49.141810 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31827dc1-eecb-41de-9708-e9ce76204eb0 None admin] RBAC: Adding query filter params (role.id=71cf67eb04704663aff719f6e3804f1a, scope.project.id=6704fc6d36e84a078282ac2057784505, user.id=9337ba85d24f434fabd701c84d5cd89a) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:49.142125 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31827dc1-eecb-41de-9708-e9ce76204eb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:49.143871 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31827dc1-eecb-41de-9708-e9ce76204eb0 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:49.151905 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 104/205] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:39:48 2022] GET /identity/v3/role_assignments?user.id=9337ba85d24f434fabd701c84d5cd89a&scope.project.id=6704fc6d36e84a078282ac2057784505&role.id=71cf67eb04704663aff719f6e3804f1a => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:49.161191 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a40897-b99b-4211-8e5d-32b8ad42945f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:49.162754 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a40897-b99b-4211-8e5d-32b8ad42945f None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:49.163271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a40897-b99b-4211-8e5d-32b8ad42945f None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:49.163766 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a40897-b99b-4211-8e5d-32b8ad42945f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:49.255085 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a40897-b99b-4211-8e5d-32b8ad42945f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:49.257013 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a40897-b99b-4211-8e5d-32b8ad42945f None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:49.277263 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:49.379649 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 102/206] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:39:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 17 16:39:49.383557 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31827dc1-eecb-41de-9708-e9ce76204eb0 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:49.499740 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31827dc1-eecb-41de-9708-e9ce76204eb0 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:49.547467 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f276faaf-e58f-4333-ab3e-88567a6da755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:49.548351 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276faaf-e58f-4333-ab3e-88567a6da755 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:49.548642 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276faaf-e58f-4333-ab3e-88567a6da755 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:49.548948 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276faaf-e58f-4333-ab3e-88567a6da755 None admin] PATH_INFO: `/v3/projects/fde6b97a4d09416a81257067f1d1b919` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:49.550815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f276faaf-e58f-4333-ab3e-88567a6da755 None admin] RBAC: Authorizing `identity:get_project(project_id=fde6b97a4d09416a81257067f1d1b919)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:49.552276 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f276faaf-e58f-4333-ab3e-88567a6da755 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:49.554257 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 105/207] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:39:49 2022] GET /identity/v3/projects/fde6b97a4d09416a81257067f1d1b919 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:49.560203 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a40897-b99b-4211-8e5d-32b8ad42945f None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:49.686127 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a40897-b99b-4211-8e5d-32b8ad42945f None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:49.704507 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:49.723573 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5591c40-efc6-4cce-b9d4-d4747182938f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:49.724260 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5591c40-efc6-4cce-b9d4-d4747182938f None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:49.724583 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5591c40-efc6-4cce-b9d4-d4747182938f None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:49.724851 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5591c40-efc6-4cce-b9d4-d4747182938f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:49.725714 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5591c40-efc6-4cce-b9d4-d4747182938f None admin] RBAC: Adding query filter params (role.id=e7249313e5334765ba16793745caed63, scope.project.id=fde6b97a4d09416a81257067f1d1b919, user.id=9eb540242e364ea7955ea951382ed935) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:49.726032 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5591c40-efc6-4cce-b9d4-d4747182938f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:49.727591 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5591c40-efc6-4cce-b9d4-d4747182938f None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:49.734211 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 103/208] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:39:49 2022] GET /identity/v3/role_assignments?user.id=9eb540242e364ea7955ea951382ed935&scope.project.id=fde6b97a4d09416a81257067f1d1b919&role.id=e7249313e5334765ba16793745caed63 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:50.266305 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624fbcb2-f2c2-464e-ac8d-83dd677adbfe None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:50.266810 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624fbcb2-f2c2-464e-ac8d-83dd677adbfe None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:50.266884 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624fbcb2-f2c2-464e-ac8d-83dd677adbfe None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:50.268530 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 106/209] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:50.274325 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd3079a-e353-4205-88e4-bb4ca00b1b3e None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:50.274806 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd3079a-e353-4205-88e4-bb4ca00b1b3e None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:50.274882 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd3079a-e353-4205-88e4-bb4ca00b1b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:50.294980 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-dcd3079a-e353-4205-88e4-bb4ca00b1b3e None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:50.296197 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-dcd3079a-e353-4205-88e4-bb4ca00b1b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:50.373374 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 104/210] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 17 16:39:50.397041 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624fbcb2-f2c2-464e-ac8d-83dd677adbfe None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:50.445219 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0f314-0999-469b-a202-4f88ca763ffe None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:50.445496 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0f314-0999-469b-a202-4f88ca763ffe None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:50.445728 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0f314-0999-469b-a202-4f88ca763ffe None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:50.447386 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 105/211] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:39:50.453053 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d30adf1-3d59-478c-9cc4-a4406f5a265c None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:50.453329 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:50.453431 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d30adf1-3d59-478c-9cc4-a4406f5a265c None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:50.453604 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d30adf1-3d59-478c-9cc4-a4406f5a265c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:50.472594 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-3d30adf1-3d59-478c-9cc4-a4406f5a265c None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:50.473846 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-3d30adf1-3d59-478c-9cc4-a4406f5a265c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:50.517616 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624fbcb2-f2c2-464e-ac8d-83dd677adbfe None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:50.548107 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 106/212] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 17 16:39:50.556418 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f3a546-5510-412a-9985-93e60a13ff60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:50.557348 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f3a546-5510-412a-9985-93e60a13ff60 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:50.557707 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f3a546-5510-412a-9985-93e60a13ff60 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:50.558060 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f3a546-5510-412a-9985-93e60a13ff60 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:50.569864 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f3a546-5510-412a-9985-93e60a13ff60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:39:50.576323 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d30adf1-3d59-478c-9cc4-a4406f5a265c None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:50.582212 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-31f3a546-5510-412a-9985-93e60a13ff60 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:39:50.582212 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:39:50.582212 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:39:50.582212 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:39:50.582212 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:39:50.582212 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:39:50.582212 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:39:50.582212 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:39:50.582212 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:39:50.582212 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:39:50.582212 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:39:50.582212 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:39:50.582212 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:39:50.582212 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:39:50.582212 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:39:50.582212 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:39:50.583152 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:39:50.583152 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:39:50.583152 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:39:50.583152 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:39:50.583152 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:39:50.583152 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:39:50.583152 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:39:50.583152 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:39:50.583152 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:39:50.583152 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:39:50.583152 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:39:50.583152 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:39:50.583152 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:39:50.583152 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:39:50.583152 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:39:50.583152 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:39:50.583933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:39:50.583933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:39:50.583933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:39:50.583933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:39:50.583933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:39:50.583933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:39:50.583933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:39:50.583933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:39:50.583933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:39:50.583933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:39:50.583933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:39:50.583933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:39:50.587445 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 107/213] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:39:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:39:50.593878 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f3a546-5510-412a-9985-93e60a13ff60 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:50.633474 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:50.708278 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d30adf1-3d59-478c-9cc4-a4406f5a265c None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:50.724486 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f3a546-5510-412a-9985-93e60a13ff60 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:50.727093 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:50.751887 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f9bdb-d84b-47bd-8151-b66e3e880b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:50.752692 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f9bdb-d84b-47bd-8151-b66e3e880b49 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:50.753011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f9bdb-d84b-47bd-8151-b66e3e880b49 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:50.753311 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f9bdb-d84b-47bd-8151-b66e3e880b49 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:50.755401 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f9bdb-d84b-47bd-8151-b66e3e880b49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:50.757194 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f9bdb-d84b-47bd-8151-b66e3e880b49 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:50.759019 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 107/214] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:39:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:50.764432 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f9bdb-d84b-47bd-8151-b66e3e880b49 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:50.775149 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f4086-824c-4764-8fbf-9c0f83aad89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:50.776328 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f4086-824c-4764-8fbf-9c0f83aad89a None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:50.776952 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f4086-824c-4764-8fbf-9c0f83aad89a None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:50.777496 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f4086-824c-4764-8fbf-9c0f83aad89a None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:50.778590 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511f4086-824c-4764-8fbf-9c0f83aad89a None admin] RBAC: Adding query filter params (name=member) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:50.779188 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511f4086-824c-4764-8fbf-9c0f83aad89a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:50.781243 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511f4086-824c-4764-8fbf-9c0f83aad89a None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:50.796590 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 108/215] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 17 16:39:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:50.797866 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f4086-824c-4764-8fbf-9c0f83aad89a None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:50.874580 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f9bdb-d84b-47bd-8151-b66e3e880b49 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:50.886691 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:50.912760 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d52b47a-c7c6-4d03-bcc7-9bd0a5a355b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:50.913845 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d52b47a-c7c6-4d03-bcc7-9bd0a5a355b3 None admin] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:50.914314 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d52b47a-c7c6-4d03-bcc7-9bd0a5a355b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:50.914774 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d52b47a-c7c6-4d03-bcc7-9bd0a5a355b3 None admin] PATH_INFO: `/v3/users` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:50.915321 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f4086-824c-4764-8fbf-9c0f83aad89a None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:50.915928 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d52b47a-c7c6-4d03-bcc7-9bd0a5a355b3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:50.917757 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d52b47a-c7c6-4d03-bcc7-9bd0a5a355b3 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:50.960241 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3590c9-a52b-4eb8-a42c-314a6f04be95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:50.961108 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3590c9-a52b-4eb8-a42c-314a6f04be95 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:50.961401 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3590c9-a52b-4eb8-a42c-314a6f04be95 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:50.961689 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3590c9-a52b-4eb8-a42c-314a6f04be95 None admin] PATH_INFO: `/v3/users/a9913fa4590a41c5818757f57ad649e4` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:50.980076 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3590c9-a52b-4eb8-a42c-314a6f04be95 None admin] RBAC: Authorizing `identity:get_user(user_id=a9913fa4590a41c5818757f57ad649e4)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:50.981764 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3590c9-a52b-4eb8-a42c-314a6f04be95 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:50.983803 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 109/216] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:39:50 2022] GET /identity/v3/users/a9913fa4590a41c5818757f57ad649e4 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:50.990069 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3590c9-a52b-4eb8-a42c-314a6f04be95 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:51.009706 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 108/217] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 17 16:39:50 2022] POST /identity/v3/users => generated 297 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:39:51.032304 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:51.130969 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3590c9-a52b-4eb8-a42c-314a6f04be95 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:51.175871 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc886ea-85e6-4a0a-a72e-806f427e4d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:51.177370 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc886ea-85e6-4a0a-a72e-806f427e4d38 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:51.178112 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc886ea-85e6-4a0a-a72e-806f427e4d38 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:51.179755 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc886ea-85e6-4a0a-a72e-806f427e4d38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:51.285330 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc886ea-85e6-4a0a-a72e-806f427e4d38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:51.287609 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc886ea-85e6-4a0a-a72e-806f427e4d38 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:51.318182 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:51.354285 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48e412-2d07-47d2-87ae-d03ee6e0960a None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:51.356352 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48e412-2d07-47d2-87ae-d03ee6e0960a None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:51.356352 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48e412-2d07-47d2-87ae-d03ee6e0960a None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:51.357838 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 109/218] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:39:51.365100 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55933d28-4056-464c-a0d5-4a2113df5a98 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:51.365508 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55933d28-4056-464c-a0d5-4a2113df5a98 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:51.365801 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55933d28-4056-464c-a0d5-4a2113df5a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:51.392186 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-55933d28-4056-464c-a0d5-4a2113df5a98 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:51.393490 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-55933d28-4056-464c-a0d5-4a2113df5a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:51.419422 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 110/219] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:39:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 17 16:39:51.426568 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc886ea-85e6-4a0a-a72e-806f427e4d38 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:51.501297 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 110/220] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 17 16:39:51.530517 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55933d28-4056-464c-a0d5-4a2113df5a98 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:51.573071 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc886ea-85e6-4a0a-a72e-806f427e4d38 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:51.604639 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:51.626927 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c7f31-db8b-47ec-9901-8002b7821a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:51.629379 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c7f31-db8b-47ec-9901-8002b7821a5b None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:51.629675 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c7f31-db8b-47ec-9901-8002b7821a5b None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:51.630002 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c7f31-db8b-47ec-9901-8002b7821a5b None admin] PATH_INFO: `/v3/projects/6704fc6d36e84a078282ac2057784505` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:51.632093 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c7f31-db8b-47ec-9901-8002b7821a5b None admin] RBAC: Authorizing `identity:get_project(project_id=6704fc6d36e84a078282ac2057784505)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:51.634912 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c7f31-db8b-47ec-9901-8002b7821a5b None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:51.639830 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 111/221] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:39:51 2022] GET /identity/v3/projects/6704fc6d36e84a078282ac2057784505 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:51.641658 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035d0b7-75a3-48b2-831f-a6a8c22aa3eb None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:51.641952 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035d0b7-75a3-48b2-831f-a6a8c22aa3eb None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:51.642235 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035d0b7-75a3-48b2-831f-a6a8c22aa3eb None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:51.648925 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 112/222] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:51.650571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035d0b7-75a3-48b2-831f-a6a8c22aa3eb None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:51.692320 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:51.703145 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55933d28-4056-464c-a0d5-4a2113df5a98 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:51.788223 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc855ee-102c-4f49-860c-9fb0d59360f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:51.789083 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc855ee-102c-4f49-860c-9fb0d59360f8 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:51.789368 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc855ee-102c-4f49-860c-9fb0d59360f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:51.789648 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc855ee-102c-4f49-860c-9fb0d59360f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:51.800068 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035d0b7-75a3-48b2-831f-a6a8c22aa3eb None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:51.805502 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc855ee-102c-4f49-860c-9fb0d59360f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:39:51.813318 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-ebc855ee-102c-4f49-860c-9fb0d59360f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:39:51.813318 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:39:51.813318 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:39:51.813318 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:39:51.813318 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:39:51.813318 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:39:51.813318 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:39:51.813318 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:39:51.813318 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:39:51.813318 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:39:51.813318 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:39:51.813318 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:39:51.813318 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:39:51.813318 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:39:51.813318 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:39:51.813318 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:39:51.814257 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:39:51.814257 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:39:51.814257 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:39:51.814257 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:39:51.814257 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:39:51.814257 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:39:51.814257 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:39:51.814257 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:39:51.814257 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:39:51.814257 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:39:51.814257 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:39:51.814257 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:39:51.814257 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:39:51.814257 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:39:51.814257 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:39:51.814257 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:39:51.815045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:39:51.815045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:39:51.815045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:39:51.815045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:39:51.815045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:39:51.815045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:39:51.815045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:39:51.815045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:39:51.815045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:39:51.815045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:39:51.815045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:39:51.815045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:39:51.815045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 111/223] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:39:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:39:51.818203 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048db15c-3fc9-4995-86d8-f42e108b9bbc None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:51.818203 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048db15c-3fc9-4995-86d8-f42e108b9bbc None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:51.818203 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048db15c-3fc9-4995-86d8-f42e108b9bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:51.841181 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-048db15c-3fc9-4995-86d8-f42e108b9bbc None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:51.842354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-048db15c-3fc9-4995-86d8-f42e108b9bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:51.881023 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f67ebf-b800-418d-a5cd-3d8c7dc85578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:51.882412 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f67ebf-b800-418d-a5cd-3d8c7dc85578 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:51.882918 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f67ebf-b800-418d-a5cd-3d8c7dc85578 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:51.883748 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f67ebf-b800-418d-a5cd-3d8c7dc85578 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:51.886737 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f67ebf-b800-418d-a5cd-3d8c7dc85578 None admin] RBAC: Adding query filter params (role.id=a47f5df8505f49419d0acb4f0f287610, scope.project.id=6704fc6d36e84a078282ac2057784505, user.id=a9913fa4590a41c5818757f57ad649e4) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:51.887307 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f67ebf-b800-418d-a5cd-3d8c7dc85578 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:51.890246 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f67ebf-b800-418d-a5cd-3d8c7dc85578 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:51.907414 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 113/224] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:39:51 2022] GET /identity/v3/role_assignments?user.id=a9913fa4590a41c5818757f57ad649e4&scope.project.id=6704fc6d36e84a078282ac2057784505&role.id=a47f5df8505f49419d0acb4f0f287610 => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:51.909059 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f67ebf-b800-418d-a5cd-3d8c7dc85578 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:51.953017 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 112/225] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:39:51.989528 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048db15c-3fc9-4995-86d8-f42e108b9bbc None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:52.054536 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:52.153483 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f67ebf-b800-418d-a5cd-3d8c7dc85578 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:52.153653 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:52.169427 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048db15c-3fc9-4995-86d8-f42e108b9bbc None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:52.207700 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b86c2-eea9-4564-94d1-18a381ab652c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:52.208587 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b86c2-eea9-4564-94d1-18a381ab652c None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:52.208890 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b86c2-eea9-4564-94d1-18a381ab652c None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:52.209167 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b86c2-eea9-4564-94d1-18a381ab652c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:52.220417 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b86c2-eea9-4564-94d1-18a381ab652c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:39:52.229683 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-339b86c2-eea9-4564-94d1-18a381ab652c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 17 16:39:52.229683 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:39:52.229683 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:39:52.229683 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:39:52.229683 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:39:52.229683 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:39:52.229683 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:39:52.229683 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:39:52.229683 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:39:52.229683 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:39:52.229683 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:39:52.229683 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:39:52.229683 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:39:52.229683 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:39:52.229683 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:39:52.229683 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:39:52.230563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:39:52.230563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:39:52.230563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:39:52.230563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:39:52.230563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:39:52.230563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:39:52.230563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:39:52.230563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:39:52.230563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:39:52.230563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:39:52.230563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:39:52.230563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:39:52.230563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:39:52.230563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:39:52.230563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:39:52.230563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:39:52.231356 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:39:52.231356 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:39:52.231356 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:39:52.231356 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:39:52.231356 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:39:52.231356 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:39:52.231356 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:39:52.231356 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:39:52.231356 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:39:52.231356 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:39:52.231356 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 17 16:39:52.231356 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:39:52.231356 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 113/226] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 17 16:39:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:39:52.236747 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b86c2-eea9-4564-94d1-18a381ab652c None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:52.236903 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c5f83f-4a63-4944-9cf6-baae288a23cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:52.238227 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5f83f-4a63-4944-9cf6-baae288a23cf None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:52.238516 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5f83f-4a63-4944-9cf6-baae288a23cf None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:52.238751 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5f83f-4a63-4944-9cf6-baae288a23cf None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:52.239528 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c5f83f-4a63-4944-9cf6-baae288a23cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:52.239837 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c5f83f-4a63-4944-9cf6-baae288a23cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:52.241271 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c5f83f-4a63-4944-9cf6-baae288a23cf None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:52.256836 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 114/227] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:39:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:52.266579 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c5f83f-4a63-4944-9cf6-baae288a23cf None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:52.344984 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b86c2-eea9-4564-94d1-18a381ab652c None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:52.384390 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74893e3a-7831-470e-a1e4-c3d28ac4e4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:52.385178 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74893e3a-7831-470e-a1e4-c3d28ac4e4a5 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:52.385378 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74893e3a-7831-470e-a1e4-c3d28ac4e4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:52.385655 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74893e3a-7831-470e-a1e4-c3d28ac4e4a5 None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:52.386439 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74893e3a-7831-470e-a1e4-c3d28ac4e4a5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:52.386731 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74893e3a-7831-470e-a1e4-c3d28ac4e4a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:52.388120 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74893e3a-7831-470e-a1e4-c3d28ac4e4a5 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:52.397645 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 114/228] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:39:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:52.403055 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74893e3a-7831-470e-a1e4-c3d28ac4e4a5 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:52.416010 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:52.506978 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c5f83f-4a63-4944-9cf6-baae288a23cf None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:52.511858 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74893e3a-7831-470e-a1e4-c3d28ac4e4a5 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:52.547946 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06034e8a-d737-44a6-b50b-aa7a1911c84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:52.548893 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06034e8a-d737-44a6-b50b-aa7a1911c84f None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:52.549182 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06034e8a-d737-44a6-b50b-aa7a1911c84f None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:52.549461 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06034e8a-d737-44a6-b50b-aa7a1911c84f None admin] PATH_INFO: `/v3/users/9337ba85d24f434fabd701c84d5cd89a` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:52.551272 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06034e8a-d737-44a6-b50b-aa7a1911c84f None admin] RBAC: Authorizing `identity:get_user(user_id=9337ba85d24f434fabd701c84d5cd89a)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:52.551687 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:52.553317 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06034e8a-d737-44a6-b50b-aa7a1911c84f None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:52.555664 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 115/229] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:39:52 2022] GET /identity/v3/users/9337ba85d24f434fabd701c84d5cd89a => generated 283 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:52.561403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06034e8a-d737-44a6-b50b-aa7a1911c84f None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:52.602872 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16479820-38a1-41d3-a7f0-cfdb56ddda64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:52.604109 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16479820-38a1-41d3-a7f0-cfdb56ddda64 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:52.604536 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16479820-38a1-41d3-a7f0-cfdb56ddda64 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:52.604982 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16479820-38a1-41d3-a7f0-cfdb56ddda64 None admin] PATH_INFO: `/v3/users/9337ba85d24f434fabd701c84d5cd89a` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:52.609137 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16479820-38a1-41d3-a7f0-cfdb56ddda64 None admin] RBAC: Authorizing `identity:get_user(user_id=9337ba85d24f434fabd701c84d5cd89a)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:52.611007 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16479820-38a1-41d3-a7f0-cfdb56ddda64 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:52.614116 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 115/230] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:39:52 2022] GET /identity/v3/users/9337ba85d24f434fabd701c84d5cd89a => generated 283 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:52.618819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16479820-38a1-41d3-a7f0-cfdb56ddda64 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:52.660831 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:52.680747 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06034e8a-d737-44a6-b50b-aa7a1911c84f None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:52.719156 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08d801-e5eb-4096-b367-da794206e31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:52.720035 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08d801-e5eb-4096-b367-da794206e31d None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:52.720318 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08d801-e5eb-4096-b367-da794206e31d None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:52.720588 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db08d801-e5eb-4096-b367-da794206e31d None admin] PATH_INFO: `/v3/projects/6704fc6d36e84a078282ac2057784505` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:52.723811 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08d801-e5eb-4096-b367-da794206e31d None admin] RBAC: Authorizing `identity:get_project(project_id=6704fc6d36e84a078282ac2057784505)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:52.725296 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db08d801-e5eb-4096-b367-da794206e31d None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:52.727027 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 116/231] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:39:52 2022] GET /identity/v3/projects/6704fc6d36e84a078282ac2057784505 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:52.733792 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08d801-e5eb-4096-b367-da794206e31d None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:52.823171 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16479820-38a1-41d3-a7f0-cfdb56ddda64 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:52.865077 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57ea58-0755-4f3f-953f-ec0ea7e37eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:52.865974 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57ea58-0755-4f3f-953f-ec0ea7e37eb4 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:52.866258 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57ea58-0755-4f3f-953f-ec0ea7e37eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:52.866537 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57ea58-0755-4f3f-953f-ec0ea7e37eb4 None admin] PATH_INFO: `/v3/projects/6704fc6d36e84a078282ac2057784505` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:52.868043 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c57ea58-0755-4f3f-953f-ec0ea7e37eb4 None admin] RBAC: Authorizing `identity:get_project(project_id=6704fc6d36e84a078282ac2057784505)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:52.870045 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c57ea58-0755-4f3f-953f-ec0ea7e37eb4 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:52.872246 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 116/232] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:39:52 2022] GET /identity/v3/projects/6704fc6d36e84a078282ac2057784505 => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:52.878064 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57ea58-0755-4f3f-953f-ec0ea7e37eb4 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:52.955522 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db08d801-e5eb-4096-b367-da794206e31d None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:52.984754 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:53.024623 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb916f25-9348-4239-a5fe-cc4a586a4596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:53.025553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb916f25-9348-4239-a5fe-cc4a586a4596 None admin] REQUEST_METHOD: `PUT` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:53.025835 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb916f25-9348-4239-a5fe-cc4a586a4596 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:53.026109 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb916f25-9348-4239-a5fe-cc4a586a4596 None admin] PATH_INFO: `/v3/projects/6704fc6d36e84a078282ac2057784505/users/9337ba85d24f434fabd701c84d5cd89a/roles/71cf67eb04704663aff719f6e3804f1a` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:53.028424 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb916f25-9348-4239-a5fe-cc4a586a4596 None admin] RBAC: Authorizing `identity:create_grant(project_id=6704fc6d36e84a078282ac2057784505, user_id=9337ba85d24f434fabd701c84d5cd89a, role_id=71cf67eb04704663aff719f6e3804f1a)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:53.030016 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb916f25-9348-4239-a5fe-cc4a586a4596 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:53.065665 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57ea58-0755-4f3f-953f-ec0ea7e37eb4 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:53.103900 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e980e-e89b-4515-a3f5-f713a1a7958e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:53.104816 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e980e-e89b-4515-a3f5-f713a1a7958e None admin] REQUEST_METHOD: `PUT` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:53.105102 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e980e-e89b-4515-a3f5-f713a1a7958e None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:53.105382 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e980e-e89b-4515-a3f5-f713a1a7958e None admin] PATH_INFO: `/v3/projects/6704fc6d36e84a078282ac2057784505/users/9337ba85d24f434fabd701c84d5cd89a/roles/1025051975f64f689effa58dc4259e36` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:53.107591 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5e980e-e89b-4515-a3f5-f713a1a7958e None admin] RBAC: Authorizing `identity:create_grant(project_id=6704fc6d36e84a078282ac2057784505, user_id=9337ba85d24f434fabd701c84d5cd89a, role_id=1025051975f64f689effa58dc4259e36)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:53.115957 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 117/233] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 17 16:39:52 2022] PUT /identity/v3/projects/6704fc6d36e84a078282ac2057784505/users/9337ba85d24f434fabd701c84d5cd89a/roles/71cf67eb04704663aff719f6e3804f1a => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 17 16:39:53.117432 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5e980e-e89b-4515-a3f5-f713a1a7958e None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:53.218991 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 117/234] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 17 16:39:52 2022] PUT /identity/v3/projects/6704fc6d36e84a078282ac2057784505/users/9337ba85d24f434fabd701c84d5cd89a/roles/1025051975f64f689effa58dc4259e36 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 17 16:39:53.546883 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be65a9-402a-4de9-9be8-d98120da36d0 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:53.547410 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be65a9-402a-4de9-9be8-d98120da36d0 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:53.547410 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be65a9-402a-4de9-9be8-d98120da36d0 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:53.548986 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 118/235] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:53.555492 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ea285-a0a6-4c5b-944d-e4a8f9369ce7 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:53.555968 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ea285-a0a6-4c5b-944d-e4a8f9369ce7 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:53.556043 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ea285-a0a6-4c5b-944d-e4a8f9369ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:53.578993 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-604ea285-a0a6-4c5b-944d-e4a8f9369ce7 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:53.580183 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-604ea285-a0a6-4c5b-944d-e4a8f9369ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:53.667404 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900269b-94ab-4a04-98e3-1805feb438fa None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:53.667872 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900269b-94ab-4a04-98e3-1805feb438fa None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:53.668043 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900269b-94ab-4a04-98e3-1805feb438fa None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:53.669553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 119/236] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:53.677351 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a6fe6-bff0-4c54-b66e-3ece18f87c9d None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:53.677670 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 118/237] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:39:53.677788 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a6fe6-bff0-4c54-b66e-3ece18f87c9d None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:53.677863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a6fe6-bff0-4c54-b66e-3ece18f87c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:53.700853 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ea285-a0a6-4c5b-944d-e4a8f9369ce7 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:53.703365 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:53.717821 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-974a6fe6-bff0-4c54-b66e-3ece18f87c9d None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:53.719134 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-974a6fe6-bff0-4c54-b66e-3ece18f87c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:53.756081 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:53.833416 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 120/238] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 17 16:39:53.856257 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ea285-a0a6-4c5b-944d-e4a8f9369ce7 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:53.873655 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974a6fe6-bff0-4c54-b66e-3ece18f87c9d None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:53.921098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573f967-8a02-490c-a4f6-13399bcd271d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:53.921912 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573f967-8a02-490c-a4f6-13399bcd271d None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:53.922194 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573f967-8a02-490c-a4f6-13399bcd271d None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:53.922473 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573f967-8a02-490c-a4f6-13399bcd271d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:53.946613 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573f967-8a02-490c-a4f6-13399bcd271d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:39:53.959105 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-e573f967-8a02-490c-a4f6-13399bcd271d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 17 16:39:53.959105 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:39:53.959105 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:39:53.959105 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:39:53.959105 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:39:53.959105 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:39:53.959105 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:39:53.959105 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:39:53.959105 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:39:53.959105 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:39:53.959105 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:39:53.959105 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:39:53.959105 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:39:53.959105 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:39:53.959105 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:39:53.959105 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:39:53.960027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:39:53.960027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:39:53.960027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:39:53.960027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:39:53.960027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:39:53.960027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:39:53.960027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:39:53.960027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:39:53.960027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:39:53.960027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:39:53.960027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:39:53.960027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:39:53.960027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:39:53.960027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:39:53.960027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:39:53.960027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:39:53.960827 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:39:53.960827 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:39:53.960827 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:39:53.960827 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:39:53.960827 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:39:53.960827 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:39:53.960827 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:39:53.960827 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:39:53.960827 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:39:53.960827 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:39:53.960827 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 17 16:39:53.960827 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:39:53.960827 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 119/239] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:39:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:39:53.967136 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573f967-8a02-490c-a4f6-13399bcd271d None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:54.084343 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573f967-8a02-490c-a4f6-13399bcd271d None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:54.086118 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:54.091068 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974a6fe6-bff0-4c54-b66e-3ece18f87c9d None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:54.121780 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdee280-1d66-4cc3-bf69-c3cd9de1888b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:54.122809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdee280-1d66-4cc3-bf69-c3cd9de1888b None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:54.123198 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdee280-1d66-4cc3-bf69-c3cd9de1888b None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:54.123583 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdee280-1d66-4cc3-bf69-c3cd9de1888b None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:54.124667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdee280-1d66-4cc3-bf69-c3cd9de1888b None admin] RBAC: Adding query filter params (name=reader) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:54.125129 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdee280-1d66-4cc3-bf69-c3cd9de1888b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:54.126677 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdee280-1d66-4cc3-bf69-c3cd9de1888b None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:54.137470 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 120/240] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 17 16:39:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:54.143185 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdee280-1d66-4cc3-bf69-c3cd9de1888b None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:54.209679 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab731c64-97c9-4f96-8944-22e5eae24cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:54.210523 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab731c64-97c9-4f96-8944-22e5eae24cbd None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:54.210814 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab731c64-97c9-4f96-8944-22e5eae24cbd None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:54.211100 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab731c64-97c9-4f96-8944-22e5eae24cbd None admin] PATH_INFO: `/v3/domains/default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:54.217523 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab731c64-97c9-4f96-8944-22e5eae24cbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:54.219172 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab731c64-97c9-4f96-8944-22e5eae24cbd None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:54.221026 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 121/241] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:39:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:54.229805 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab731c64-97c9-4f96-8944-22e5eae24cbd None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:54.239241 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:54.255960 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdee280-1d66-4cc3-bf69-c3cd9de1888b None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:54.293700 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e1ce6-cd05-4fbe-83e1-4f50554bd905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:54.294584 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e1ce6-cd05-4fbe-83e1-4f50554bd905 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:54.294874 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e1ce6-cd05-4fbe-83e1-4f50554bd905 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:54.295156 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e1ce6-cd05-4fbe-83e1-4f50554bd905 None admin] PATH_INFO: `/v3/users/2ed3cd3cfa3e43028bfd692a0be0a033` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:54.309426 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8e1ce6-cd05-4fbe-83e1-4f50554bd905 None admin] RBAC: Authorizing `identity:get_user(user_id=2ed3cd3cfa3e43028bfd692a0be0a033)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:54.310907 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8e1ce6-cd05-4fbe-83e1-4f50554bd905 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:54.312624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 121/242] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:39:54 2022] GET /identity/v3/users/2ed3cd3cfa3e43028bfd692a0be0a033 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:54.318777 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e1ce6-cd05-4fbe-83e1-4f50554bd905 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:54.346061 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:54.426758 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e1ce6-cd05-4fbe-83e1-4f50554bd905 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:54.460022 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab731c64-97c9-4f96-8944-22e5eae24cbd None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:54.464655 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d22f95b-ce1e-455a-adbb-cd9d7c2128a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:54.465364 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22f95b-ce1e-455a-adbb-cd9d7c2128a8 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:54.465652 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22f95b-ce1e-455a-adbb-cd9d7c2128a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:54.465932 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22f95b-ce1e-455a-adbb-cd9d7c2128a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:54.511643 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef598ef-4b2c-4b9f-a342-fb992b21863d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:54.512668 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef598ef-4b2c-4b9f-a342-fb992b21863d None admin] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:54.513629 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef598ef-4b2c-4b9f-a342-fb992b21863d None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:54.514104 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef598ef-4b2c-4b9f-a342-fb992b21863d None admin] PATH_INFO: `/v3/users` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:54.515442 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef598ef-4b2c-4b9f-a342-fb992b21863d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:54.517630 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef598ef-4b2c-4b9f-a342-fb992b21863d None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:54.546863 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d22f95b-ce1e-455a-adbb-cd9d7c2128a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:54.548739 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d22f95b-ce1e-455a-adbb-cd9d7c2128a8 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:54.578077 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:54.621494 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 122/243] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 17 16:39:54 2022] POST /identity/v3/users => generated 293 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:39:54.623092 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985dfb7-fe48-4da2-a070-6ec5e7f767ca None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:54.623390 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985dfb7-fe48-4da2-a070-6ec5e7f767ca None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:54.623682 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985dfb7-fe48-4da2-a070-6ec5e7f767ca None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:54.631041 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 123/244] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:54 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:54.637915 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f8733-e175-4b03-a84e-25c680bfce47 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:54.638205 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f8733-e175-4b03-a84e-25c680bfce47 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:54.638486 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f8733-e175-4b03-a84e-25c680bfce47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:54.676595 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-4a0f8733-e175-4b03-a84e-25c680bfce47 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:54.682746 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-4a0f8733-e175-4b03-a84e-25c680bfce47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:54.719916 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 122/245] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:39:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 17 16:39:54.726627 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d22f95b-ce1e-455a-adbb-cd9d7c2128a8 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:54.803469 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:54.828345 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 124/246] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:39:54.871821 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f8733-e175-4b03-a84e-25c680bfce47 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:54.891395 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d22f95b-ce1e-455a-adbb-cd9d7c2128a8 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:54.927820 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:54.948582 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f95b3-588c-401b-85db-564a52f9e439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:54.949936 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f95b3-588c-401b-85db-564a52f9e439 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:54.952840 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f95b3-588c-401b-85db-564a52f9e439 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:54.953083 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f95b3-588c-401b-85db-564a52f9e439 None admin] PATH_INFO: `/v3/projects/6704fc6d36e84a078282ac2057784505` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:54.954644 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007f95b3-588c-401b-85db-564a52f9e439 None admin] RBAC: Authorizing `identity:get_project(project_id=6704fc6d36e84a078282ac2057784505)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:54.956285 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007f95b3-588c-401b-85db-564a52f9e439 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:54.958829 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 123/247] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:39:54 2022] GET /identity/v3/projects/6704fc6d36e84a078282ac2057784505 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:54.966622 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f95b3-588c-401b-85db-564a52f9e439 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:55.046596 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f8733-e175-4b03-a84e-25c680bfce47 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:55.097372 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac35ff55-0e51-4462-8978-b037a77d2354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:55.099373 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35ff55-0e51-4462-8978-b037a77d2354 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:55.099890 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35ff55-0e51-4462-8978-b037a77d2354 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:55.100365 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35ff55-0e51-4462-8978-b037a77d2354 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:55.116229 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f95b3-588c-401b-85db-564a52f9e439 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:55.117085 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac35ff55-0e51-4462-8978-b037a77d2354 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:39:55.126608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-ac35ff55-0e51-4462-8978-b037a77d2354 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:39:55.126608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:39:55.126608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:39:55.126608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:39:55.126608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:39:55.126608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:39:55.126608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:39:55.126608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:39:55.126608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:39:55.126608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:39:55.126608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:39:55.126608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:39:55.126608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:39:55.126608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:39:55.126608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:39:55.126608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:39:55.127522 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:39:55.127522 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:39:55.127522 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:39:55.127522 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:39:55.127522 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:39:55.127522 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:39:55.127522 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:39:55.127522 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:39:55.127522 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:39:55.127522 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:39:55.127522 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:39:55.127522 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:39:55.127522 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:39:55.127522 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:39:55.127522 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:39:55.127522 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:39:55.128584 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:39:55.128584 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:39:55.128584 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:39:55.128584 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:39:55.128584 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:39:55.128584 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:39:55.128584 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:39:55.128584 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:39:55.128584 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:39:55.128584 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:39:55.128584 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:39:55.128584 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:39:55.131393 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 125/248] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:39:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:39:55.137408 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac35ff55-0e51-4462-8978-b037a77d2354 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:55.173539 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9afa1-4f36-454b-bf70-37fa20c41e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:55.174451 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9afa1-4f36-454b-bf70-37fa20c41e90 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:55.177220 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9afa1-4f36-454b-bf70-37fa20c41e90 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:55.177666 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9afa1-4f36-454b-bf70-37fa20c41e90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:55.179013 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf9afa1-4f36-454b-bf70-37fa20c41e90 None admin] RBAC: Adding query filter params (role.id=e7249313e5334765ba16793745caed63, scope.project.id=6704fc6d36e84a078282ac2057784505, user.id=2ed3cd3cfa3e43028bfd692a0be0a033) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:55.179013 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf9afa1-4f36-454b-bf70-37fa20c41e90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:55.181022 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf9afa1-4f36-454b-bf70-37fa20c41e90 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:55.193989 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 124/249] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:39:54 2022] GET /identity/v3/role_assignments?user.id=2ed3cd3cfa3e43028bfd692a0be0a033&scope.project.id=6704fc6d36e84a078282ac2057784505&role.id=e7249313e5334765ba16793745caed63 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:55.225436 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:55.287625 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac35ff55-0e51-4462-8978-b037a77d2354 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:55.335388 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd53a44-71c6-4852-b1e7-d0664ae6a4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:55.336531 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd53a44-71c6-4852-b1e7-d0664ae6a4a4 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:55.336884 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd53a44-71c6-4852-b1e7-d0664ae6a4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:55.337154 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd53a44-71c6-4852-b1e7-d0664ae6a4a4 None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:55.337941 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd53a44-71c6-4852-b1e7-d0664ae6a4a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:55.338251 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd53a44-71c6-4852-b1e7-d0664ae6a4a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:55.339685 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd53a44-71c6-4852-b1e7-d0664ae6a4a4 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:55.349295 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 126/250] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 17 16:39:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:55.355555 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9afa1-4f36-454b-bf70-37fa20c41e90 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:55.421851 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:55.476878 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9afa1-4f36-454b-bf70-37fa20c41e90 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:55.518520 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14408f99-dc19-49df-a1fa-271ae07725be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:55.519432 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14408f99-dc19-49df-a1fa-271ae07725be None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:55.519719 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14408f99-dc19-49df-a1fa-271ae07725be None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:55.519999 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14408f99-dc19-49df-a1fa-271ae07725be None admin] PATH_INFO: `/v3/users/a9913fa4590a41c5818757f57ad649e4` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:55.521291 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14408f99-dc19-49df-a1fa-271ae07725be None admin] RBAC: Authorizing `identity:get_user(user_id=a9913fa4590a41c5818757f57ad649e4)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:55.522754 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14408f99-dc19-49df-a1fa-271ae07725be None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:55.524493 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 125/251] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:39:55 2022] GET /identity/v3/users/a9913fa4590a41c5818757f57ad649e4 => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:55.533935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd53a44-71c6-4852-b1e7-d0664ae6a4a4 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:55.626833 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0ff19-e9c7-45d5-92a1-018786d87a4b None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:55.627868 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0ff19-e9c7-45d5-92a1-018786d87a4b None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:55.629241 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb0ff19-e9c7-45d5-92a1-018786d87a4b None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:55.632489 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 126/252] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:39:55.637297 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4cbec-1ace-45f4-8fb8-6ee3c0455351 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:55.637835 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4cbec-1ace-45f4-8fb8-6ee3c0455351 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:55.638312 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4cbec-1ace-45f4-8fb8-6ee3c0455351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:55.657390 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-fcd4cbec-1ace-45f4-8fb8-6ee3c0455351 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:55.658990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-fcd4cbec-1ace-45f4-8fb8-6ee3c0455351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:55.737165 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd53a44-71c6-4852-b1e7-d0664ae6a4a4 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:55.762465 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:55.806185 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 127/253] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 17 16:39:55.807748 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815473a-46c8-442c-8e20-8f4a542f588c None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:55.808039 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815473a-46c8-442c-8e20-8f4a542f588c None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:55.808326 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815473a-46c8-442c-8e20-8f4a542f588c None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:55.809681 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 128/254] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:55.811405 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9159ee1c-c62d-4d83-9c46-429bbfca6a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:55.812516 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159ee1c-c62d-4d83-9c46-429bbfca6a86 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:55.813037 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159ee1c-c62d-4d83-9c46-429bbfca6a86 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:55.815006 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae5c10-568d-42b4-8f3b-7343bb20856c None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:55.815300 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae5c10-568d-42b4-8f3b-7343bb20856c None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:55.816159 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae5c10-568d-42b4-8f3b-7343bb20856c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:55.816294 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159ee1c-c62d-4d83-9c46-429bbfca6a86 None admin] PATH_INFO: `/v3/projects/6704fc6d36e84a078282ac2057784505` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:55.820105 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9159ee1c-c62d-4d83-9c46-429bbfca6a86 None admin] RBAC: Authorizing `identity:get_project(project_id=6704fc6d36e84a078282ac2057784505)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:55.822100 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9159ee1c-c62d-4d83-9c46-429bbfca6a86 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:55.824069 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 127/255] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:39:55 2022] GET /identity/v3/projects/6704fc6d36e84a078282ac2057784505 => generated 302 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:55.829969 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9159ee1c-c62d-4d83-9c46-429bbfca6a86 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:55.836807 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-aeae5c10-568d-42b4-8f3b-7343bb20856c None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:55.836807 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-aeae5c10-568d-42b4-8f3b-7343bb20856c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:55.954370 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9159ee1c-c62d-4d83-9c46-429bbfca6a86 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:55.967606 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:55.999594 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 129/256] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:39:56.000415 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae5c10-568d-42b4-8f3b-7343bb20856c None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:56.012021 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16652c2-14a3-42cd-9b93-67550c080ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:56.012766 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16652c2-14a3-42cd-9b93-67550c080ee6 None admin] REQUEST_METHOD: `PUT` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:56.013078 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16652c2-14a3-42cd-9b93-67550c080ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:56.013360 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16652c2-14a3-42cd-9b93-67550c080ee6 None admin] PATH_INFO: `/v3/projects/6704fc6d36e84a078282ac2057784505/users/a9913fa4590a41c5818757f57ad649e4/roles/a47f5df8505f49419d0acb4f0f287610` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:56.025713 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16652c2-14a3-42cd-9b93-67550c080ee6 None admin] RBAC: Authorizing `identity:create_grant(project_id=6704fc6d36e84a078282ac2057784505, user_id=a9913fa4590a41c5818757f57ad649e4, role_id=a47f5df8505f49419d0acb4f0f287610)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:56.027215 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16652c2-14a3-42cd-9b93-67550c080ee6 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:56.103826 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 128/257] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 17 16:39:55 2022] PUT /identity/v3/projects/6704fc6d36e84a078282ac2057784505/users/a9913fa4590a41c5818757f57ad649e4/roles/a47f5df8505f49419d0acb4f0f287610 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 17 16:39:56.104468 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16652c2-14a3-42cd-9b93-67550c080ee6 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:56.184159 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:56.199209 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae5c10-568d-42b4-8f3b-7343bb20856c None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:56.275082 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3a1c4-425b-4990-afbd-6ad184a867c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:56.275916 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3a1c4-425b-4990-afbd-6ad184a867c0 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:56.276206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3a1c4-425b-4990-afbd-6ad184a867c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:56.276488 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3a1c4-425b-4990-afbd-6ad184a867c0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:56.302106 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d3a1c4-425b-4990-afbd-6ad184a867c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:39:56.314329 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-d5d3a1c4-425b-4990-afbd-6ad184a867c0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 17 16:39:56.314329 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:39:56.314329 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:39:56.314329 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:39:56.314329 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:39:56.314329 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:39:56.314329 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:39:56.314329 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:39:56.314329 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:39:56.314329 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:39:56.314329 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:39:56.314329 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:39:56.314329 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:39:56.314329 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:39:56.314329 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:39:56.314329 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:39:56.315261 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:39:56.315261 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:39:56.315261 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:39:56.315261 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:39:56.315261 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:39:56.315261 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:39:56.315261 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:39:56.315261 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:39:56.315261 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:39:56.315261 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:39:56.315261 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:39:56.315261 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:39:56.315261 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:39:56.315261 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:39:56.315261 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:39:56.315261 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:39:56.316176 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:39:56.316176 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:39:56.316176 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:39:56.316176 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:39:56.316176 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:39:56.316176 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:39:56.316176 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:39:56.316176 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:39:56.316176 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:39:56.316176 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:39:56.316176 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 17 16:39:56.316176 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:39:56.316176 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 130/258] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 17 16:39:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:39:56.325213 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3a1c4-425b-4990-afbd-6ad184a867c0 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:56.329138 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16652c2-14a3-42cd-9b93-67550c080ee6 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:56.406660 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5de81b-e12a-4bae-ba79-1caaeb23793a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:56.413103 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5de81b-e12a-4bae-ba79-1caaeb23793a None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:56.413623 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5de81b-e12a-4bae-ba79-1caaeb23793a None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:56.414100 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5de81b-e12a-4bae-ba79-1caaeb23793a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:56.424706 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:56.435062 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5de81b-e12a-4bae-ba79-1caaeb23793a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:39:56.446357 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-0f5de81b-e12a-4bae-ba79-1caaeb23793a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:39:56.446357 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:39:56.446357 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:39:56.446357 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:39:56.446357 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:39:56.446357 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:39:56.446357 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:39:56.446357 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:39:56.446357 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:39:56.446357 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:39:56.446357 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:39:56.446357 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:39:56.446357 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:39:56.446357 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:39:56.446357 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:39:56.446357 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:39:56.447236 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:39:56.447236 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:39:56.447236 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:39:56.447236 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:39:56.447236 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:39:56.447236 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:39:56.447236 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:39:56.447236 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:39:56.447236 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:39:56.447236 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:39:56.447236 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:39:56.447236 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:39:56.447236 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:39:56.447236 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:39:56.447236 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:39:56.447236 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:39:56.448023 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:39:56.448023 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:39:56.448023 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:39:56.448023 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:39:56.448023 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:39:56.448023 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:39:56.448023 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:39:56.448023 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:39:56.448023 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:39:56.448023 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:39:56.448023 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:39:56.448023 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:39:56.456850 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 129/259] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:39:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:39:56.473979 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5de81b-e12a-4bae-ba79-1caaeb23793a None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:56.567780 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3a1c4-425b-4990-afbd-6ad184a867c0 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:56.587075 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:56.590697 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5de81b-e12a-4bae-ba79-1caaeb23793a None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:56.636626 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab591da-f9e6-40f9-9d14-a84176f647b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:56.637686 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab591da-f9e6-40f9-9d14-a84176f647b7 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:56.638140 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab591da-f9e6-40f9-9d14-a84176f647b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:56.638612 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab591da-f9e6-40f9-9d14-a84176f647b7 None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:56.639550 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab591da-f9e6-40f9-9d14-a84176f647b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:56.640024 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab591da-f9e6-40f9-9d14-a84176f647b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:56.641658 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab591da-f9e6-40f9-9d14-a84176f647b7 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:56.657263 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 130/260] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:39:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:56.663692 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab591da-f9e6-40f9-9d14-a84176f647b7 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:56.684830 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627303bb-9e5c-4477-94a6-8634b972171d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:56.684830 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627303bb-9e5c-4477-94a6-8634b972171d None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:56.684830 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627303bb-9e5c-4477-94a6-8634b972171d None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:56.684830 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627303bb-9e5c-4477-94a6-8634b972171d None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:56.685533 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627303bb-9e5c-4477-94a6-8634b972171d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:56.685533 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627303bb-9e5c-4477-94a6-8634b972171d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:56.685533 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627303bb-9e5c-4477-94a6-8634b972171d None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:56.700384 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 131/261] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:39:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:56.705980 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627303bb-9e5c-4477-94a6-8634b972171d None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:56.787400 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab591da-f9e6-40f9-9d14-a84176f647b7 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:56.826798 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89207f42-ff42-4744-abd8-1188cec3d400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:56.827924 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89207f42-ff42-4744-abd8-1188cec3d400 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:56.828356 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89207f42-ff42-4744-abd8-1188cec3d400 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:56.828819 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89207f42-ff42-4744-abd8-1188cec3d400 None admin] PATH_INFO: `/v3/users/9337ba85d24f434fabd701c84d5cd89a` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:56.830314 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89207f42-ff42-4744-abd8-1188cec3d400 None admin] RBAC: Authorizing `identity:get_user(user_id=9337ba85d24f434fabd701c84d5cd89a)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:56.831966 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89207f42-ff42-4744-abd8-1188cec3d400 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:56.834298 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 131/262] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:39:56 2022] GET /identity/v3/users/9337ba85d24f434fabd701c84d5cd89a => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:56.839907 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89207f42-ff42-4744-abd8-1188cec3d400 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:56.872588 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:56.958869 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89207f42-ff42-4744-abd8-1188cec3d400 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:56.968699 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627303bb-9e5c-4477-94a6-8634b972171d None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:56.973290 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:56.998622 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff2413-ad99-48aa-a30d-577529cde620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:56.999822 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff2413-ad99-48aa-a30d-577529cde620 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:57.000256 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff2413-ad99-48aa-a30d-577529cde620 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:57.001190 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff2413-ad99-48aa-a30d-577529cde620 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:57.057624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124314e8-27cb-4dcf-abf1-1b10289c7a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:57.058527 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124314e8-27cb-4dcf-abf1-1b10289c7a0a None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:57.058809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124314e8-27cb-4dcf-abf1-1b10289c7a0a None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:57.059083 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124314e8-27cb-4dcf-abf1-1b10289c7a0a None admin] PATH_INFO: `/v3/users/9337ba85d24f434fabd701c84d5cd89a` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:57.060328 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124314e8-27cb-4dcf-abf1-1b10289c7a0a None admin] RBAC: Authorizing `identity:get_user(user_id=9337ba85d24f434fabd701c84d5cd89a)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:57.061818 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124314e8-27cb-4dcf-abf1-1b10289c7a0a None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:57.063763 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 132/263] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:39:56 2022] GET /identity/v3/users/9337ba85d24f434fabd701c84d5cd89a => generated 283 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:57.070214 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124314e8-27cb-4dcf-abf1-1b10289c7a0a None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:57.081314 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff2413-ad99-48aa-a30d-577529cde620 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:57.083034 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff2413-ad99-48aa-a30d-577529cde620 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:57.113515 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:57.261879 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 132/264] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:39:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 17 16:39:57.267609 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff2413-ad99-48aa-a30d-577529cde620 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:57.300843 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124314e8-27cb-4dcf-abf1-1b10289c7a0a None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:57.366613 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc2f62-222d-4707-9f18-a6f2f00529bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:57.367197 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc2f62-222d-4707-9f18-a6f2f00529bf None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:57.367481 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc2f62-222d-4707-9f18-a6f2f00529bf None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:57.367761 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc2f62-222d-4707-9f18-a6f2f00529bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:57.433674 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:57.475733 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc2f62-222d-4707-9f18-a6f2f00529bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:57.477265 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc2f62-222d-4707-9f18-a6f2f00529bf None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:57.527030 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff2413-ad99-48aa-a30d-577529cde620 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:57.598293 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:57.616530 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2859c6-0fd1-46d2-b201-87674023f42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:57.617833 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2859c6-0fd1-46d2-b201-87674023f42b None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:57.618069 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2859c6-0fd1-46d2-b201-87674023f42b None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:57.618347 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2859c6-0fd1-46d2-b201-87674023f42b None admin] PATH_INFO: `/v3/projects/6704fc6d36e84a078282ac2057784505` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:57.619658 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2859c6-0fd1-46d2-b201-87674023f42b None admin] RBAC: Authorizing `identity:get_project(project_id=6704fc6d36e84a078282ac2057784505)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:57.621233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2859c6-0fd1-46d2-b201-87674023f42b None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:57.624939 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 133/265] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:39:57 2022] GET /identity/v3/projects/6704fc6d36e84a078282ac2057784505 => generated 302 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:57.631060 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2859c6-0fd1-46d2-b201-87674023f42b None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:57.635183 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 133/266] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:39:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 17 16:39:57.641452 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc2f62-222d-4707-9f18-a6f2f00529bf None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:57.816812 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:57.856274 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2859c6-0fd1-46d2-b201-87674023f42b None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:57.881756 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc2f62-222d-4707-9f18-a6f2f00529bf None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:57.935523 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92970fa8-1127-4331-9c7d-194648c9d8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:57.936616 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92970fa8-1127-4331-9c7d-194648c9d8aa None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:57.937124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92970fa8-1127-4331-9c7d-194648c9d8aa None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:57.937627 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92970fa8-1127-4331-9c7d-194648c9d8aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:57.938671 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92970fa8-1127-4331-9c7d-194648c9d8aa None admin] RBAC: Adding query filter params (role.id=1025051975f64f689effa58dc4259e36, scope.project.id=6704fc6d36e84a078282ac2057784505, user.id=9337ba85d24f434fabd701c84d5cd89a) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:57.939291 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92970fa8-1127-4331-9c7d-194648c9d8aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:57.941373 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92970fa8-1127-4331-9c7d-194648c9d8aa None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:57.953350 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 134/267] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:39:57 2022] GET /identity/v3/role_assignments?user.id=9337ba85d24f434fabd701c84d5cd89a&scope.project.id=6704fc6d36e84a078282ac2057784505&role.id=1025051975f64f689effa58dc4259e36 => generated 608 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:57.955299 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83b4bb-bb69-424e-8f41-4122fa1d5a01 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:57.957285 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c6bcc-fe9c-41dd-ad86-f0abb2f6d452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:57.958475 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c6bcc-fe9c-41dd-ad86-f0abb2f6d452 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:57.959003 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c6bcc-fe9c-41dd-ad86-f0abb2f6d452 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:57.959524 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c6bcc-fe9c-41dd-ad86-f0abb2f6d452 None admin] PATH_INFO: `/v3/projects/6704fc6d36e84a078282ac2057784505` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:57.960915 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83b4bb-bb69-424e-8f41-4122fa1d5a01 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:57.961442 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83b4bb-bb69-424e-8f41-4122fa1d5a01 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:57.963177 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 135/268] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:57 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:57.964284 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241c6bcc-fe9c-41dd-ad86-f0abb2f6d452 None admin] RBAC: Authorizing `identity:get_project(project_id=6704fc6d36e84a078282ac2057784505)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:57.972701 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241c6bcc-fe9c-41dd-ad86-f0abb2f6d452 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:57.977111 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43617c15-ae18-470d-ae99-2baadc689951 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:57.978251 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 134/269] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:39:57 2022] GET /identity/v3/projects/6704fc6d36e84a078282ac2057784505 => generated 302 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:57.978795 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43617c15-ae18-470d-ae99-2baadc689951 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:57.979286 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43617c15-ae18-470d-ae99-2baadc689951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:57.993306 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c6bcc-fe9c-41dd-ad86-f0abb2f6d452 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:58.028215 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-43617c15-ae18-470d-ae99-2baadc689951 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:58.029594 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-43617c15-ae18-470d-ae99-2baadc689951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:58.190703 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:58.210075 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 136/270] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:39:58.211533 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7df2e-78ac-493a-a837-3d4b16c6c1d4 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:58.211851 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7df2e-78ac-493a-a837-3d4b16c6c1d4 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:58.212085 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7df2e-78ac-493a-a837-3d4b16c6c1d4 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:58.213524 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 137/271] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:58.219974 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd647220-6695-4557-847c-b8f8945a682c None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:58.220258 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd647220-6695-4557-847c-b8f8945a682c None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:58.220537 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd647220-6695-4557-847c-b8f8945a682c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:58.234064 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c6bcc-fe9c-41dd-ad86-f0abb2f6d452 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:58.245231 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-dd647220-6695-4557-847c-b8f8945a682c None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:58.247059 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-dd647220-6695-4557-847c-b8f8945a682c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:58.287686 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da63ce29-ff77-487b-be01-827d0edffc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:58.288384 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63ce29-ff77-487b-be01-827d0edffc2e None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:58.288670 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63ce29-ff77-487b-be01-827d0edffc2e None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:58.288985 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63ce29-ff77-487b-be01-827d0edffc2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:58.289878 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da63ce29-ff77-487b-be01-827d0edffc2e None admin] RBAC: Adding query filter params (role.id=71cf67eb04704663aff719f6e3804f1a, scope.project.id=6704fc6d36e84a078282ac2057784505, user.id=9337ba85d24f434fabd701c84d5cd89a) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:58.290195 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da63ce29-ff77-487b-be01-827d0edffc2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:58.291727 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da63ce29-ff77-487b-be01-827d0edffc2e None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:58.302785 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 135/272] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:39:57 2022] GET /identity/v3/role_assignments?user.id=9337ba85d24f434fabd701c84d5cd89a&scope.project.id=6704fc6d36e84a078282ac2057784505&role.id=71cf67eb04704663aff719f6e3804f1a => generated 608 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:58.304716 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da63ce29-ff77-487b-be01-827d0edffc2e None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:58.311887 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:58.400135 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 138/273] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:39:58.401893 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092722ef-ee09-4f00-b3c0-a9c346667b64 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:58.402187 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092722ef-ee09-4f00-b3c0-a9c346667b64 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:58.402535 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092722ef-ee09-4f00-b3c0-a9c346667b64 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:58.404668 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 139/274] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:58.411977 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3355177-4d77-4f18-99ab-f27bff10bc43 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:58.412408 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3355177-4d77-4f18-99ab-f27bff10bc43 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:58.412816 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3355177-4d77-4f18-99ab-f27bff10bc43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:58.443550 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-c3355177-4d77-4f18-99ab-f27bff10bc43 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:58.445147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-c3355177-4d77-4f18-99ab-f27bff10bc43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:58.479893 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da63ce29-ff77-487b-be01-827d0edffc2e None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:58.529042 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578cfd7-dee9-4f47-8981-347b15c46fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:58.530151 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578cfd7-dee9-4f47-8981-347b15c46fd1 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:58.530446 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578cfd7-dee9-4f47-8981-347b15c46fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:58.530727 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578cfd7-dee9-4f47-8981-347b15c46fd1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:58.544341 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e578cfd7-dee9-4f47-8981-347b15c46fd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:39:58.549073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 140/275] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:39:58.550995 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3355177-4d77-4f18-99ab-f27bff10bc43 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:58.560717 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-e578cfd7-dee9-4f47-8981-347b15c46fd1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:39:58.560717 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:39:58.560717 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:39:58.560717 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:39:58.560717 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:39:58.560717 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:39:58.560717 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:39:58.560717 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:39:58.560717 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:39:58.560717 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:39:58.560717 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:39:58.560717 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:39:58.560717 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:39:58.560717 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:39:58.560717 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:39:58.560717 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:39:58.561689 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:39:58.561689 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:39:58.561689 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:39:58.561689 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:39:58.561689 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:39:58.561689 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:39:58.561689 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:39:58.561689 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:39:58.561689 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:39:58.561689 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:39:58.561689 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:39:58.561689 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:39:58.561689 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:39:58.561689 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:39:58.561689 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:39:58.561689 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:39:58.562467 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:39:58.562467 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:39:58.562467 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:39:58.562467 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:39:58.562467 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:39:58.562467 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:39:58.562467 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:39:58.562467 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:39:58.562467 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:39:58.562467 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:39:58.562467 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:39:58.562467 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:39:58.562467 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 136/276] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:39:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:39:58.571699 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578cfd7-dee9-4f47-8981-347b15c46fd1 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:58.639094 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:58.678382 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:58.699048 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3355177-4d77-4f18-99ab-f27bff10bc43 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:58.724278 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578cfd7-dee9-4f47-8981-347b15c46fd1 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:58.746658 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f92f6a-e3b8-420a-a0d8-bb3a28a568e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:58.747474 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f92f6a-e3b8-420a-a0d8-bb3a28a568e2 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:58.747766 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f92f6a-e3b8-420a-a0d8-bb3a28a568e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:58.748080 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f92f6a-e3b8-420a-a0d8-bb3a28a568e2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:58.760406 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f92f6a-e3b8-420a-a0d8-bb3a28a568e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:39:58.765164 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978bbc4b-9c31-46ef-8869-8bec78a58227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:58.766573 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978bbc4b-9c31-46ef-8869-8bec78a58227 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:58.767287 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978bbc4b-9c31-46ef-8869-8bec78a58227 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:58.767980 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978bbc4b-9c31-46ef-8869-8bec78a58227 None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:58.768277 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-09f92f6a-e3b8-420a-a0d8-bb3a28a568e2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 17 16:39:58.768277 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:39:58.768277 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:39:58.768277 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:39:58.768277 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:39:58.768277 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:39:58.768277 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:39:58.768277 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:39:58.768277 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:39:58.768277 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:39:58.768277 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:39:58.768277 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:39:58.768277 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:39:58.768277 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:39:58.768277 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:39:58.768277 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:39:58.770320 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:39:58.770320 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:39:58.770320 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:39:58.770320 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:39:58.770320 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:39:58.770320 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:39:58.770320 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:39:58.770320 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:39:58.770320 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:39:58.770320 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:39:58.770320 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:39:58.770320 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:39:58.770320 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:39:58.770320 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:39:58.770320 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:39:58.770320 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:39:58.771763 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:39:58.771763 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:39:58.771763 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:39:58.771763 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:39:58.771763 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:39:58.771763 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:39:58.771763 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:39:58.771763 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:39:58.771763 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:39:58.771763 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:39:58.771763 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 17 16:39:58.771763 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:39:58.771763 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 141/277] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:39:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:39:58.772958 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978bbc4b-9c31-46ef-8869-8bec78a58227 None admin] RBAC: Adding query filter params (name=member) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:58.772958 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978bbc4b-9c31-46ef-8869-8bec78a58227 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:58.773155 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f92f6a-e3b8-420a-a0d8-bb3a28a568e2 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:58.773265 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978bbc4b-9c31-46ef-8869-8bec78a58227 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:58.784820 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 137/278] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 17 16:39:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:58.784820 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978bbc4b-9c31-46ef-8869-8bec78a58227 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:58.901239 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978bbc4b-9c31-46ef-8869-8bec78a58227 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:58.904061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f92f6a-e3b8-420a-a0d8-bb3a28a568e2 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:58.916563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:58.932568 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:58.942716 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc7a27-2b2c-49e3-999f-f71e8c223bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:58.943406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc7a27-2b2c-49e3-999f-f71e8c223bec None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:58.943693 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc7a27-2b2c-49e3-999f-f71e8c223bec None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:58.943812 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6880fc36-dbb7-43fe-9b6d-6763dfd55f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:58.944015 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fc7a27-2b2c-49e3-999f-f71e8c223bec None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:58.944541 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880fc36-dbb7-43fe-9b6d-6763dfd55f38 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:58.944740 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc7a27-2b2c-49e3-999f-f71e8c223bec None admin] RBAC: Adding query filter params (name=reader) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:58.945092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc7a27-2b2c-49e3-999f-f71e8c223bec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:58.945282 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880fc36-dbb7-43fe-9b6d-6763dfd55f38 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:58.946127 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6880fc36-dbb7-43fe-9b6d-6763dfd55f38 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:58.946749 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fc7a27-2b2c-49e3-999f-f71e8c223bec None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:58.948394 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6880fc36-dbb7-43fe-9b6d-6763dfd55f38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:58.955886 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6880fc36-dbb7-43fe-9b6d-6763dfd55f38 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:58.955886 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 142/279] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:39:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:58.955886 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6880fc36-dbb7-43fe-9b6d-6763dfd55f38 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:58.958858 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 138/280] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 17 16:39:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:58.959941 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc7a27-2b2c-49e3-999f-f71e8c223bec None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:59.072707 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6880fc36-dbb7-43fe-9b6d-6763dfd55f38 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:59.079555 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fc7a27-2b2c-49e3-999f-f71e8c223bec None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:59.109321 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602e4e6-74df-4f53-94a1-99de8df41906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:59.110222 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602e4e6-74df-4f53-94a1-99de8df41906 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:59.110497 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602e4e6-74df-4f53-94a1-99de8df41906 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:59.110777 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602e4e6-74df-4f53-94a1-99de8df41906 None admin] PATH_INFO: `/v3/users/180f5f3585c241a2993e43d06b130d7d` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:59.116196 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d084be-763f-407c-bce2-3704e245bd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:59.116896 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d084be-763f-407c-bce2-3704e245bd14 None admin] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:59.117181 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d084be-763f-407c-bce2-3704e245bd14 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:59.117459 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d084be-763f-407c-bce2-3704e245bd14 None admin] PATH_INFO: `/v3/users` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:59.118170 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d084be-763f-407c-bce2-3704e245bd14 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:59.119615 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d084be-763f-407c-bce2-3704e245bd14 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:59.125456 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d602e4e6-74df-4f53-94a1-99de8df41906 None admin] RBAC: Authorizing `identity:get_user(user_id=180f5f3585c241a2993e43d06b130d7d)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:59.127192 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d602e4e6-74df-4f53-94a1-99de8df41906 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:59.129778 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 143/281] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:39:58 2022] GET /identity/v3/users/180f5f3585c241a2993e43d06b130d7d => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:59.130604 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602e4e6-74df-4f53-94a1-99de8df41906 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:59.183165 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:59.228874 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 139/282] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 17 16:39:58 2022] POST /identity/v3/users => generated 293 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:39:59.228874 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d084be-763f-407c-bce2-3704e245bd14 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:59.274415 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602e4e6-74df-4f53-94a1-99de8df41906 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:59.286080 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:59.329107 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77750e86-f222-43b9-8176-8addd3944736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:59.330221 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77750e86-f222-43b9-8176-8addd3944736 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:59.330612 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77750e86-f222-43b9-8176-8addd3944736 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:59.331552 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77750e86-f222-43b9-8176-8addd3944736 None admin] PATH_INFO: `/v3/users/2ed3cd3cfa3e43028bfd692a0be0a033` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:59.333398 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77750e86-f222-43b9-8176-8addd3944736 None admin] RBAC: Authorizing `identity:get_user(user_id=2ed3cd3cfa3e43028bfd692a0be0a033)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:59.335346 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77750e86-f222-43b9-8176-8addd3944736 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:59.337052 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 144/283] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:39:59 2022] GET /identity/v3/users/2ed3cd3cfa3e43028bfd692a0be0a033 => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:59.342938 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77750e86-f222-43b9-8176-8addd3944736 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:59.361033 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d084be-763f-407c-bce2-3704e245bd14 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:59.406042 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdf57aa-edd9-44ea-a9dd-6b9e4d3dc1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:59.408206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf57aa-edd9-44ea-a9dd-6b9e4d3dc1d3 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:59.408206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf57aa-edd9-44ea-a9dd-6b9e4d3dc1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:59.408206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdf57aa-edd9-44ea-a9dd-6b9e4d3dc1d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:59.408206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdf57aa-edd9-44ea-a9dd-6b9e4d3dc1d3 None admin] RBAC: Adding query filter params (role.id=a47f5df8505f49419d0acb4f0f287610, user.id=180f5f3585c241a2993e43d06b130d7d, scope.system=all) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:59.408501 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdf57aa-edd9-44ea-a9dd-6b9e4d3dc1d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:59.412062 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdf57aa-edd9-44ea-a9dd-6b9e4d3dc1d3 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:59.416852 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 140/284] 10.222.0.40 () {56 vars in 1308 bytes} [Sun Jul 17 16:39:59 2022] GET /identity/v3/role_assignments?user.id=180f5f3585c241a2993e43d06b130d7d&scope.system=all&role.id=a47f5df8505f49419d0acb4f0f287610 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:59.418094 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea256c46-31a9-4e09-bedc-35e771724ed7 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:59.418388 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea256c46-31a9-4e09-bedc-35e771724ed7 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:59.418670 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea256c46-31a9-4e09-bedc-35e771724ed7 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:59.419936 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 141/285] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:39:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:39:59.428629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87bf9e4-524a-4b23-93c5-a852ed3b96ed None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:59.428629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87bf9e4-524a-4b23-93c5-a852ed3b96ed None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:59.428629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87bf9e4-524a-4b23-93c5-a852ed3b96ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:59.450159 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-e87bf9e4-524a-4b23-93c5-a852ed3b96ed None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:39:59.451614 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-e87bf9e4-524a-4b23-93c5-a852ed3b96ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:39:59.478081 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77750e86-f222-43b9-8176-8addd3944736 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:59.482941 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:59.533377 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74c424-ce7a-4f7c-8f94-2b2d657a2f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:59.534970 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74c424-ce7a-4f7c-8f94-2b2d657a2f99 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:59.535389 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74c424-ce7a-4f7c-8f94-2b2d657a2f99 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:59.535676 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74c424-ce7a-4f7c-8f94-2b2d657a2f99 None admin] PATH_INFO: `/v3/projects/6704fc6d36e84a078282ac2057784505` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:59.537363 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd74c424-ce7a-4f7c-8f94-2b2d657a2f99 None admin] RBAC: Authorizing `identity:get_project(project_id=6704fc6d36e84a078282ac2057784505)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:59.538889 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd74c424-ce7a-4f7c-8f94-2b2d657a2f99 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:59.544822 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 145/286] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:39:59 2022] GET /identity/v3/projects/6704fc6d36e84a078282ac2057784505 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:39:59.546993 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74c424-ce7a-4f7c-8f94-2b2d657a2f99 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:59.549349 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 142/287] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:39:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 17 16:39:59.574051 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87bf9e4-524a-4b23-93c5-a852ed3b96ed None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:59.630900 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:59.676836 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74c424-ce7a-4f7c-8f94-2b2d657a2f99 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:59.713697 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87bf9e4-524a-4b23-93c5-a852ed3b96ed None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:59.724609 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a228345-f23c-4295-8f0e-4b3cab5c960f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:59.725559 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a228345-f23c-4295-8f0e-4b3cab5c960f None admin] REQUEST_METHOD: `PUT` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:59.725850 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a228345-f23c-4295-8f0e-4b3cab5c960f None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:59.726125 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a228345-f23c-4295-8f0e-4b3cab5c960f None admin] PATH_INFO: `/v3/projects/6704fc6d36e84a078282ac2057784505/users/2ed3cd3cfa3e43028bfd692a0be0a033/roles/e7249313e5334765ba16793745caed63` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:59.729056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a228345-f23c-4295-8f0e-4b3cab5c960f None admin] RBAC: Authorizing `identity:create_grant(project_id=6704fc6d36e84a078282ac2057784505, user_id=2ed3cd3cfa3e43028bfd692a0be0a033, role_id=e7249313e5334765ba16793745caed63)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:59.731081 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a228345-f23c-4295-8f0e-4b3cab5c960f None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:59.757023 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550955e9-fcf0-4ee1-850b-b5187855a121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:59.757763 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550955e9-fcf0-4ee1-850b-b5187855a121 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:59.758063 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550955e9-fcf0-4ee1-850b-b5187855a121 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:59.758347 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550955e9-fcf0-4ee1-850b-b5187855a121 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:59.770370 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550955e9-fcf0-4ee1-850b-b5187855a121 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:39:59.779044 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-550955e9-fcf0-4ee1-850b-b5187855a121 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:39:59.779044 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:39:59.779044 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:39:59.779044 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:39:59.779044 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:39:59.779044 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:39:59.779044 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:39:59.779044 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:39:59.779044 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:39:59.779044 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:39:59.779044 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:39:59.779044 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:39:59.779044 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:39:59.779044 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:39:59.779044 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:39:59.779044 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:39:59.780015 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:39:59.780015 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:39:59.780015 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:39:59.780015 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:39:59.780015 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:39:59.780015 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:39:59.780015 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:39:59.780015 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:39:59.780015 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:39:59.780015 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:39:59.780015 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:39:59.780015 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:39:59.780015 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:39:59.780015 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:39:59.780015 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:39:59.780015 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:39:59.780836 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:39:59.780836 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:39:59.780836 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:39:59.780836 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:39:59.780836 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:39:59.780836 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:39:59.780836 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:39:59.780836 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:39:59.780836 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:39:59.780836 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:39:59.780836 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:39:59.780836 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:39:59.780836 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 143/288] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:39:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:39:59.788325 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550955e9-fcf0-4ee1-850b-b5187855a121 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:39:59.807324 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 146/289] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 17 16:39:59 2022] PUT /identity/v3/projects/6704fc6d36e84a078282ac2057784505/users/2ed3cd3cfa3e43028bfd692a0be0a033/roles/e7249313e5334765ba16793745caed63 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 17 16:39:59.916851 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:39:59.919004 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550955e9-fcf0-4ee1-850b-b5187855a121 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:39:59.960004 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd17145-e672-4bf0-9e6f-baf02282d2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:39:59.960691 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd17145-e672-4bf0-9e6f-baf02282d2d2 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:39:59.961019 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd17145-e672-4bf0-9e6f-baf02282d2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:39:59.961305 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd17145-e672-4bf0-9e6f-baf02282d2d2 None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:39:59.962101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd17145-e672-4bf0-9e6f-baf02282d2d2 None admin] RBAC: Adding query filter params (name=member) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:39:59.962422 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd17145-e672-4bf0-9e6f-baf02282d2d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:39:59.963867 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd17145-e672-4bf0-9e6f-baf02282d2d2 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:39:59.974201 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 144/290] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 17 16:39:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:39:59.981955 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a228345-f23c-4295-8f0e-4b3cab5c960f None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:00.044353 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:00.123213 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a228345-f23c-4295-8f0e-4b3cab5c960f None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:00.160821 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d508a0c1-2a23-480d-9da2-88d55f488679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:00.161717 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508a0c1-2a23-480d-9da2-88d55f488679 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:00.162053 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508a0c1-2a23-480d-9da2-88d55f488679 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:00.162332 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d508a0c1-2a23-480d-9da2-88d55f488679 None admin] PATH_INFO: `/v3/users/a9913fa4590a41c5818757f57ad649e4` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:00.163542 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d508a0c1-2a23-480d-9da2-88d55f488679 None admin] RBAC: Authorizing `identity:get_user(user_id=a9913fa4590a41c5818757f57ad649e4)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:00.165038 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d508a0c1-2a23-480d-9da2-88d55f488679 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:00.166712 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 147/291] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:39:59 2022] GET /identity/v3/users/a9913fa4590a41c5818757f57ad649e4 => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:00.173469 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd17145-e672-4bf0-9e6f-baf02282d2d2 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:00.341618 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd17145-e672-4bf0-9e6f-baf02282d2d2 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:00.397120 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01f1930-aa43-4646-961b-63fee35168f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:00.397795 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f1930-aa43-4646-961b-63fee35168f6 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:00.398085 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f1930-aa43-4646-961b-63fee35168f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:00.398373 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01f1930-aa43-4646-961b-63fee35168f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:00.438695 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:00.482170 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01f1930-aa43-4646-961b-63fee35168f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:00.483707 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01f1930-aa43-4646-961b-63fee35168f6 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:00.601907 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 145/292] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:40:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 17 16:40:00.607765 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d508a0c1-2a23-480d-9da2-88d55f488679 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:00.738148 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d508a0c1-2a23-480d-9da2-88d55f488679 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:00.742407 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:00.779183 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c471178-6f97-4b2a-8b98-115f6b30f6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:00.780070 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c471178-6f97-4b2a-8b98-115f6b30f6e5 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:00.780349 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c471178-6f97-4b2a-8b98-115f6b30f6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:00.780624 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c471178-6f97-4b2a-8b98-115f6b30f6e5 None admin] PATH_INFO: `/v3/projects/6704fc6d36e84a078282ac2057784505` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:00.781856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c471178-6f97-4b2a-8b98-115f6b30f6e5 None admin] RBAC: Authorizing `identity:get_project(project_id=6704fc6d36e84a078282ac2057784505)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:00.783306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c471178-6f97-4b2a-8b98-115f6b30f6e5 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:00.786176 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 148/293] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:40:00 2022] GET /identity/v3/projects/6704fc6d36e84a078282ac2057784505 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:00.794329 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01f1930-aa43-4646-961b-63fee35168f6 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:00.902030 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:00.964656 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01f1930-aa43-4646-961b-63fee35168f6 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:01.003507 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf50e64-03a7-454b-bbd8-d1c59bcb4009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:01.004285 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf50e64-03a7-454b-bbd8-d1c59bcb4009 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:01.004578 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf50e64-03a7-454b-bbd8-d1c59bcb4009 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:01.005002 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf50e64-03a7-454b-bbd8-d1c59bcb4009 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:01.006238 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf50e64-03a7-454b-bbd8-d1c59bcb4009 None admin] RBAC: Adding query filter params (role.id=a47f5df8505f49419d0acb4f0f287610, scope.project.id=6704fc6d36e84a078282ac2057784505, user.id=a9913fa4590a41c5818757f57ad649e4) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:01.006735 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf50e64-03a7-454b-bbd8-d1c59bcb4009 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:01.008921 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf50e64-03a7-454b-bbd8-d1c59bcb4009 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:01.016007 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 146/294] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:40:00 2022] GET /identity/v3/role_assignments?user.id=a9913fa4590a41c5818757f57ad649e4&scope.project.id=6704fc6d36e84a078282ac2057784505&role.id=a47f5df8505f49419d0acb4f0f287610 => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:01.868654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6303ef4-658f-42f7-aaac-b2d97b2f2861 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:01.869710 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6303ef4-658f-42f7-aaac-b2d97b2f2861 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:01.870206 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6303ef4-658f-42f7-aaac-b2d97b2f2861 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:01.872037 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 149/295] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:01.878095 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515a7a7-4f5a-4291-95cd-3462f397281b None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:01.878819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515a7a7-4f5a-4291-95cd-3462f397281b None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:01.879298 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515a7a7-4f5a-4291-95cd-3462f397281b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:01.897420 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-2515a7a7-4f5a-4291-95cd-3462f397281b None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:01.898974 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-2515a7a7-4f5a-4291-95cd-3462f397281b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:01.918822 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9d28b-a3e7-4a9c-8a78-7afbc9e7aae0 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:01.919453 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9d28b-a3e7-4a9c-8a78-7afbc9e7aae0 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:01.919928 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b9d28b-a3e7-4a9c-8a78-7afbc9e7aae0 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:01.921767 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 150/296] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:01.931630 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d8cb4-8a8d-4ffa-8f23-94f4573bec62 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:01.932225 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d8cb4-8a8d-4ffa-8f23-94f4573bec62 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:01.932728 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d8cb4-8a8d-4ffa-8f23-94f4573bec62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:01.950835 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-f08d8cb4-8a8d-4ffa-8f23-94f4573bec62 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:01.952499 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-f08d8cb4-8a8d-4ffa-8f23-94f4573bec62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:01.987671 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 147/297] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:02.011736 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2515a7a7-4f5a-4291-95cd-3462f397281b None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:02.035413 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 151/298] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:02.049946 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb3e1a-50a2-4bed-b9c5-0b94f2fc3c4e None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:02.050252 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb3e1a-50a2-4bed-b9c5-0b94f2fc3c4e None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:02.050701 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb3e1a-50a2-4bed-b9c5-0b94f2fc3c4e None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:02.052492 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 152/299] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:40:02.057992 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06604c42-82d2-40c0-a5fe-776395a6587b None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:02.058283 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06604c42-82d2-40c0-a5fe-776395a6587b None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:02.058561 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06604c42-82d2-40c0-a5fe-776395a6587b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:02.075208 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:02.076640 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-06604c42-82d2-40c0-a5fe-776395a6587b None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:02.077859 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-06604c42-82d2-40c0-a5fe-776395a6587b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:02.126389 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2515a7a7-4f5a-4291-95cd-3462f397281b None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:02.150286 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 153/300] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:02.161803 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06604c42-82d2-40c0-a5fe-776395a6587b None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:02.177374 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adef627-645c-410e-b512-5649ffc12d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:02.178701 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adef627-645c-410e-b512-5649ffc12d2a None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:02.178988 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adef627-645c-410e-b512-5649ffc12d2a None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:02.179277 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adef627-645c-410e-b512-5649ffc12d2a None admin] PATH_INFO: `/v3/domains/default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:02.180506 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adef627-645c-410e-b512-5649ffc12d2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:02.182038 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adef627-645c-410e-b512-5649ffc12d2a None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:02.188039 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 148/301] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:40:02.188039 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adef627-645c-410e-b512-5649ffc12d2a None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:02.204041 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:02.285247 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06604c42-82d2-40c0-a5fe-776395a6587b None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:02.291765 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adef627-645c-410e-b512-5649ffc12d2a None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:02.313989 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:02.320442 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9bcdfe-db60-4a82-9016-259903107058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:02.323981 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bcdfe-db60-4a82-9016-259903107058 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:02.323981 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bcdfe-db60-4a82-9016-259903107058 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:02.323981 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bcdfe-db60-4a82-9016-259903107058 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:02.326293 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a22802d-a1c2-4638-b8c3-2205826b9791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:02.327088 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22802d-a1c2-4638-b8c3-2205826b9791 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:02.327377 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22802d-a1c2-4638-b8c3-2205826b9791 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:02.327650 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a22802d-a1c2-4638-b8c3-2205826b9791 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:02.333369 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9bcdfe-db60-4a82-9016-259903107058 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:02.340209 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a22802d-a1c2-4638-b8c3-2205826b9791 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:02.341147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-7d9bcdfe-db60-4a82-9016-259903107058 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 17 16:40:02.341147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:02.341147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:02.341147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:02.341147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:02.341147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:02.341147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:02.341147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:02.341147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:02.341147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:02.341147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:02.341147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:02.341147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:02.341147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:02.341147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:02.341147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:02.342056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:02.342056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:02.342056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:02.342056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:02.342056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:02.342056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:02.342056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:02.342056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:02.342056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:02.342056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:02.342056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:02.342056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:02.342056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:02.342056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:02.342056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:02.342056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:02.342893 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:02.342893 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:02.342893 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:02.342893 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:02.342893 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:02.342893 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:02.342893 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:02.342893 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:02.342893 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:02.342893 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:02.342893 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 17 16:40:02.342893 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:02.342893 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 154/302] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:40:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:02.343552 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9bcdfe-db60-4a82-9016-259903107058 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:02.350853 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-8a22802d-a1c2-4638-b8c3-2205826b9791 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:40:02.350853 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:02.350853 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:02.350853 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:02.350853 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:02.350853 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:02.350853 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:02.350853 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:02.350853 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:02.350853 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:02.350853 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:02.350853 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:02.350853 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:02.350853 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:02.350853 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:02.350853 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:02.351803 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:02.351803 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:02.351803 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:02.351803 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:02.351803 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:02.351803 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:02.351803 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:02.351803 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:02.351803 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:02.351803 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:02.351803 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:02.351803 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:02.351803 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:02.351803 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:02.351803 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:02.351803 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:02.352856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:02.352856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:02.352856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:02.352856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:02.352856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:02.352856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:02.352856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:02.352856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:02.352856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:02.352856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:02.352856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:40:02.352856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:02.352856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 149/303] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:40:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:02.353753 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09fc0b-bb94-46c3-9119-1e8689593ffe None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:02.354017 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09fc0b-bb94-46c3-9119-1e8689593ffe None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:02.354297 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd09fc0b-bb94-46c3-9119-1e8689593ffe None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:02.359928 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 150/304] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:02.359928 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd09fc0b-bb94-46c3-9119-1e8689593ffe None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:02.438783 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:02.450788 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9bcdfe-db60-4a82-9016-259903107058 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:02.457659 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd09fc0b-bb94-46c3-9119-1e8689593ffe None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:02.485730 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5479a-965d-4cdd-a98d-ac937a75bee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:02.486411 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5479a-965d-4cdd-a98d-ac937a75bee1 None admin] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:02.486952 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5479a-965d-4cdd-a98d-ac937a75bee1 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:02.487062 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5479a-965d-4cdd-a98d-ac937a75bee1 None admin] PATH_INFO: `/v3/groups` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:02.488029 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff5479a-965d-4cdd-a98d-ac937a75bee1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:02.489528 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff5479a-965d-4cdd-a98d-ac937a75bee1 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:02.491324 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e500d6-8236-47b2-b66d-7d3b0ab5b19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:02.491999 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e500d6-8236-47b2-b66d-7d3b0ab5b19f None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:02.492347 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e500d6-8236-47b2-b66d-7d3b0ab5b19f None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:02.492632 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e500d6-8236-47b2-b66d-7d3b0ab5b19f None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:02.493531 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e500d6-8236-47b2-b66d-7d3b0ab5b19f None admin] RBAC: Adding query filter params (name=reader) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:02.493844 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e500d6-8236-47b2-b66d-7d3b0ab5b19f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:02.495231 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e500d6-8236-47b2-b66d-7d3b0ab5b19f None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:02.503579 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 151/305] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 17 16:40:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:02.504390 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e500d6-8236-47b2-b66d-7d3b0ab5b19f None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:02.549069 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:02.554251 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 155/306] 10.222.0.40 () {60 vars in 1134 bytes} [Sun Jul 17 16:40:02 2022] POST /identity/v3/groups => generated 229 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:40:02.555629 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959e3d0-aa48-4d92-922d-b2d7f58d9b93 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:02.555929 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959e3d0-aa48-4d92-922d-b2d7f58d9b93 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:02.556424 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e959e3d0-aa48-4d92-922d-b2d7f58d9b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:02.587757 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-e959e3d0-aa48-4d92-922d-b2d7f58d9b93 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:02.589059 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-e959e3d0-aa48-4d92-922d-b2d7f58d9b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:02.611877 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e500d6-8236-47b2-b66d-7d3b0ab5b19f None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:02.654068 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42daa5f-defc-4be0-891d-9ef84d87de7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:02.654772 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42daa5f-defc-4be0-891d-9ef84d87de7c None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:02.655061 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42daa5f-defc-4be0-891d-9ef84d87de7c None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:02.655344 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42daa5f-defc-4be0-891d-9ef84d87de7c None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:02.656151 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42daa5f-defc-4be0-891d-9ef84d87de7c None admin] RBAC: Adding query filter params (name=member) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:02.656466 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42daa5f-defc-4be0-891d-9ef84d87de7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:02.657967 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42daa5f-defc-4be0-891d-9ef84d87de7c None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:02.672517 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 152/307] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 17 16:40:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:02.672517 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42daa5f-defc-4be0-891d-9ef84d87de7c None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:02.675287 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 156/308] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:02.677804 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e959e3d0-aa48-4d92-922d-b2d7f58d9b93 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:02.776730 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:02.800952 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e959e3d0-aa48-4d92-922d-b2d7f58d9b93 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:02.804152 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42daa5f-defc-4be0-891d-9ef84d87de7c None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:02.813085 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:02.836263 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9a04d-e667-40e9-9bb6-883189ad9a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:02.837217 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9a04d-e667-40e9-9bb6-883189ad9a3a None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:02.837545 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9a04d-e667-40e9-9bb6-883189ad9a3a None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:02.837783 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9a04d-e667-40e9-9bb6-883189ad9a3a None admin] PATH_INFO: `/v3/users/180f5f3585c241a2993e43d06b130d7d` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:02.838973 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a9a04d-e667-40e9-9bb6-883189ad9a3a None admin] RBAC: Authorizing `identity:get_user(user_id=180f5f3585c241a2993e43d06b130d7d)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:02.839873 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e93ccb5-e7bd-45d5-bac9-3d3c909e3a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:02.840446 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a9a04d-e667-40e9-9bb6-883189ad9a3a None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:02.840756 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e93ccb5-e7bd-45d5-bac9-3d3c909e3a4b None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:02.841087 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e93ccb5-e7bd-45d5-bac9-3d3c909e3a4b None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:02.841371 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e93ccb5-e7bd-45d5-bac9-3d3c909e3a4b None admin] PATH_INFO: `/v3/users/8e00631bc0db4adaba3c76db98febe9e` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:02.842143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 157/309] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:40:02 2022] GET /identity/v3/users/180f5f3585c241a2993e43d06b130d7d => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:02.842958 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9a04d-e667-40e9-9bb6-883189ad9a3a None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:02.861645 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e93ccb5-e7bd-45d5-bac9-3d3c909e3a4b None admin] RBAC: Authorizing `identity:get_user(user_id=8e00631bc0db4adaba3c76db98febe9e)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:02.865462 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e93ccb5-e7bd-45d5-bac9-3d3c909e3a4b None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:02.871096 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 153/310] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:40:02 2022] GET /identity/v3/users/8e00631bc0db4adaba3c76db98febe9e => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:02.871096 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e93ccb5-e7bd-45d5-bac9-3d3c909e3a4b None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:02.961083 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9a04d-e667-40e9-9bb6-883189ad9a3a None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:02.980062 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e93ccb5-e7bd-45d5-bac9-3d3c909e3a4b None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:02.998721 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823a61c-3f73-4b77-ac8d-59578cf49822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:02.999527 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823a61c-3f73-4b77-ac8d-59578cf49822 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:02.999811 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823a61c-3f73-4b77-ac8d-59578cf49822 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:03.000087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823a61c-3f73-4b77-ac8d-59578cf49822 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:03.012080 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823a61c-3f73-4b77-ac8d-59578cf49822 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:03.019059 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec06e62-a8f1-44fd-a4d4-6184e197a6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:03.019955 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec06e62-a8f1-44fd-a4d4-6184e197a6e7 None admin] REQUEST_METHOD: `PUT` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:03.020245 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec06e62-a8f1-44fd-a4d4-6184e197a6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:03.020467 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-b823a61c-3f73-4b77-ac8d-59578cf49822 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 17 16:40:03.020467 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:03.020467 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:03.020467 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:03.020467 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:03.020467 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:03.020467 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:03.020467 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:03.020467 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:03.020467 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:03.020467 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:03.020467 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:03.020467 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:03.020467 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:03.020467 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:03.020467 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:03.021354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:03.021354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:03.021354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:03.021354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:03.021354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:03.021354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:03.021354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:03.021354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:03.021354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:03.021354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:03.021354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:03.021354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:03.021354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:03.021354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:03.021354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:03.021354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:03.022153 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:03.022153 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:03.022153 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:03.022153 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:03.022153 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:03.022153 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:03.022153 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:03.022153 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:03.022153 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:03.022153 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:03.022153 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 17 16:40:03.022153 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:03.022748 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec06e62-a8f1-44fd-a4d4-6184e197a6e7 None admin] PATH_INFO: `/v3/system/users/180f5f3585c241a2993e43d06b130d7d/roles/a47f5df8505f49419d0acb4f0f287610` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:03.023464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 158/311] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:40:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:03.023464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823a61c-3f73-4b77-ac8d-59578cf49822 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:03.024617 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec06e62-a8f1-44fd-a4d4-6184e197a6e7 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=180f5f3585c241a2993e43d06b130d7d, role_id=a47f5df8505f49419d0acb4f0f287610)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:03.026048 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec06e62-a8f1-44fd-a4d4-6184e197a6e7 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:03.035049 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:03.109554 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 154/312] 10.222.0.40 () {58 vars in 1334 bytes} [Sun Jul 17 16:40:02 2022] PUT /identity/v3/system/users/180f5f3585c241a2993e43d06b130d7d/roles/a47f5df8505f49419d0acb4f0f287610 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 17 16:40:03.110261 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec06e62-a8f1-44fd-a4d4-6184e197a6e7 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:03.137555 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823a61c-3f73-4b77-ac8d-59578cf49822 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:03.158931 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:03.183007 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a551f37-826e-453f-b601-152671780741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:03.183767 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a551f37-826e-453f-b601-152671780741 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:03.184060 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a551f37-826e-453f-b601-152671780741 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:03.184341 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a551f37-826e-453f-b601-152671780741 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:03.185333 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a551f37-826e-453f-b601-152671780741 None admin] RBAC: Adding query filter params (role.id=e7249313e5334765ba16793745caed63, user.id=8e00631bc0db4adaba3c76db98febe9e, scope.system=all) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:03.185656 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a551f37-826e-453f-b601-152671780741 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:03.187216 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a551f37-826e-453f-b601-152671780741 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:03.194346 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 159/313] 10.222.0.40 () {56 vars in 1308 bytes} [Sun Jul 17 16:40:03 2022] GET /identity/v3/role_assignments?user.id=8e00631bc0db4adaba3c76db98febe9e&scope.system=all&role.id=e7249313e5334765ba16793745caed63 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:03.267870 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec06e62-a8f1-44fd-a4d4-6184e197a6e7 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:03.310793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f9cd3-2778-45d7-bc8d-124de05df8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:03.311749 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f9cd3-2778-45d7-bc8d-124de05df8e8 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:03.312188 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f9cd3-2778-45d7-bc8d-124de05df8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:03.312615 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f9cd3-2778-45d7-bc8d-124de05df8e8 None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:03.313587 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3f9cd3-2778-45d7-bc8d-124de05df8e8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:03.314069 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3f9cd3-2778-45d7-bc8d-124de05df8e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:03.316571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3f9cd3-2778-45d7-bc8d-124de05df8e8 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:03.331719 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 155/314] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 17 16:40:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:03.333027 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a551f37-826e-453f-b601-152671780741 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:03.413300 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:03.446825 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a551f37-826e-453f-b601-152671780741 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:03.483423 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d48c3-a223-4ee2-a264-40bdade52fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:03.484321 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d48c3-a223-4ee2-a264-40bdade52fe6 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:03.484608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d48c3-a223-4ee2-a264-40bdade52fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:03.484907 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d48c3-a223-4ee2-a264-40bdade52fe6 None admin] PATH_INFO: `/v3/users/2ed3cd3cfa3e43028bfd692a0be0a033` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:03.486131 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d48c3-a223-4ee2-a264-40bdade52fe6 None admin] RBAC: Authorizing `identity:get_user(user_id=2ed3cd3cfa3e43028bfd692a0be0a033)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:03.488211 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d48c3-a223-4ee2-a264-40bdade52fe6 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:03.490300 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 160/315] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:40:03 2022] GET /identity/v3/users/2ed3cd3cfa3e43028bfd692a0be0a033 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:03.495542 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f9cd3-2778-45d7-bc8d-124de05df8e8 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:03.605882 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:03.609909 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f9cd3-2778-45d7-bc8d-124de05df8e8 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:03.647489 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908bcd90-8912-4231-abed-175b941dbbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:03.648157 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bcd90-8912-4231-abed-175b941dbbf3 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:03.648450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bcd90-8912-4231-abed-175b941dbbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:03.648740 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908bcd90-8912-4231-abed-175b941dbbf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:03.733492 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908bcd90-8912-4231-abed-175b941dbbf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:03.735522 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908bcd90-8912-4231-abed-175b941dbbf3 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:03.841676 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:03.853514 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 156/316] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:40:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 17 16:40:03.864897 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d48c3-a223-4ee2-a264-40bdade52fe6 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:03.979256 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d48c3-a223-4ee2-a264-40bdade52fe6 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:04.017867 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa757859-77da-4ee4-a657-686f9285cf72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:04.018808 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa757859-77da-4ee4-a657-686f9285cf72 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:04.019096 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa757859-77da-4ee4-a657-686f9285cf72 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:04.019373 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa757859-77da-4ee4-a657-686f9285cf72 None admin] PATH_INFO: `/v3/projects/6704fc6d36e84a078282ac2057784505` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:04.020643 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa757859-77da-4ee4-a657-686f9285cf72 None admin] RBAC: Authorizing `identity:get_project(project_id=6704fc6d36e84a078282ac2057784505)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:04.022231 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa757859-77da-4ee4-a657-686f9285cf72 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:04.024357 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 161/317] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:40:03 2022] GET /identity/v3/projects/6704fc6d36e84a078282ac2057784505 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:04.029929 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908bcd90-8912-4231-abed-175b941dbbf3 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:04.138630 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908bcd90-8912-4231-abed-175b941dbbf3 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:04.177064 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d98f25-f85c-410e-aad0-dd1874e00c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:04.177792 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d98f25-f85c-410e-aad0-dd1874e00c29 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:04.178152 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d98f25-f85c-410e-aad0-dd1874e00c29 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:04.178349 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d98f25-f85c-410e-aad0-dd1874e00c29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:04.179197 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d98f25-f85c-410e-aad0-dd1874e00c29 None admin] RBAC: Adding query filter params (role.id=e7249313e5334765ba16793745caed63, scope.project.id=6704fc6d36e84a078282ac2057784505, user.id=2ed3cd3cfa3e43028bfd692a0be0a033) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:04.179506 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d98f25-f85c-410e-aad0-dd1874e00c29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:04.181030 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d98f25-f85c-410e-aad0-dd1874e00c29 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:04.187738 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 157/318] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:40:04 2022] GET /identity/v3/role_assignments?user.id=2ed3cd3cfa3e43028bfd692a0be0a033&scope.project.id=6704fc6d36e84a078282ac2057784505&role.id=e7249313e5334765ba16793745caed63 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:04.708584 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481e172-7597-4998-9c26-c2ff4d5e369d None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:04.708998 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481e172-7597-4998-9c26-c2ff4d5e369d None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:04.709197 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481e172-7597-4998-9c26-c2ff4d5e369d None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:04.710738 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 162/319] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:04.715684 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a23f4-c48c-42e4-96c0-e2b13c50f7db None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:04.715977 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a23f4-c48c-42e4-96c0-e2b13c50f7db None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:04.716261 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a23f4-c48c-42e4-96c0-e2b13c50f7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:04.734122 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-656a23f4-c48c-42e4-96c0-e2b13c50f7db None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:04.735317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-656a23f4-c48c-42e4-96c0-e2b13c50f7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:04.790342 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:04.808440 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 158/320] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:04.832218 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2481e172-7597-4998-9c26-c2ff4d5e369d None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:04.842396 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:04.941496 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2481e172-7597-4998-9c26-c2ff4d5e369d None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:04.977525 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a5e85e-dd6b-459b-8e70-1070f0a7b17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:04.978258 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5e85e-dd6b-459b-8e70-1070f0a7b17b None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:04.978546 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5e85e-dd6b-459b-8e70-1070f0a7b17b None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:04.978828 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5e85e-dd6b-459b-8e70-1070f0a7b17b None admin] PATH_INFO: `/v3/domains/default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:04.980017 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a5e85e-dd6b-459b-8e70-1070f0a7b17b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:04.981561 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a5e85e-dd6b-459b-8e70-1070f0a7b17b None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:04.988747 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 163/321] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:04.990000 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a23f4-c48c-42e4-96c0-e2b13c50f7db None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:05.099103 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a23f4-c48c-42e4-96c0-e2b13c50f7db None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:05.133746 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385b25b-9bdd-4f52-a2c0-68884ce4c4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:05.134425 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385b25b-9bdd-4f52-a2c0-68884ce4c4d7 None admin] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:05.134709 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385b25b-9bdd-4f52-a2c0-68884ce4c4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:05.134986 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385b25b-9bdd-4f52-a2c0-68884ce4c4d7 None admin] PATH_INFO: `/v3/groups` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:05.135673 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3385b25b-9bdd-4f52-a2c0-68884ce4c4d7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:05.137150 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3385b25b-9bdd-4f52-a2c0-68884ce4c4d7 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:05.225869 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41253d-205b-4d77-8d6f-fa805f4847e8 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:05.226243 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41253d-205b-4d77-8d6f-fa805f4847e8 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:05.226474 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41253d-205b-4d77-8d6f-fa805f4847e8 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:05.227935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 164/322] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:40:05.233063 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530bf07-7c2f-45e6-a9e3-69ef74474267 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:05.233350 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530bf07-7c2f-45e6-a9e3-69ef74474267 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:05.233650 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1530bf07-7c2f-45e6-a9e3-69ef74474267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:05.246207 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 159/323] 10.222.0.40 () {60 vars in 1134 bytes} [Sun Jul 17 16:40:04 2022] POST /identity/v3/groups => generated 226 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 17 16:40:05.251036 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-1530bf07-7c2f-45e6-a9e3-69ef74474267 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:05.252272 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-1530bf07-7c2f-45e6-a9e3-69ef74474267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:05.331508 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 165/324] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 17 16:40:05.354885 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385b25b-9bdd-4f52-a2c0-68884ce4c4d7 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:05.391870 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac33f6-cda1-446f-a61c-f8bf6a33189d None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:05.392183 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac33f6-cda1-446f-a61c-f8bf6a33189d None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:05.392443 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac33f6-cda1-446f-a61c-f8bf6a33189d None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:05.394428 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 166/325] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:05.400436 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32834862-1f2a-4a40-b3be-8a8dfa2486d6 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:05.400732 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32834862-1f2a-4a40-b3be-8a8dfa2486d6 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:05.401047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32834862-1f2a-4a40-b3be-8a8dfa2486d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:05.407299 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:05.408300 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:05.421723 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-32834862-1f2a-4a40-b3be-8a8dfa2486d6 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:05.422923 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-32834862-1f2a-4a40-b3be-8a8dfa2486d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:05.470914 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385b25b-9bdd-4f52-a2c0-68884ce4c4d7 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:05.501488 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 167/326] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 17 16:40:05.520446 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86de04c-00e9-40d5-819d-809a83cb453a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:05.521356 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86de04c-00e9-40d5-819d-809a83cb453a None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:05.521643 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86de04c-00e9-40d5-819d-809a83cb453a None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:05.522020 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86de04c-00e9-40d5-819d-809a83cb453a None admin] PATH_INFO: `/v3/roles/member` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:05.526814 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32834862-1f2a-4a40-b3be-8a8dfa2486d6 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:05.538938 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86de04c-00e9-40d5-819d-809a83cb453a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:05.547677 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-a86de04c-00e9-40d5-819d-809a83cb453a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:40:05.547677 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:05.547677 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:05.547677 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:05.547677 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:05.547677 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:05.547677 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:05.547677 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:05.547677 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:05.547677 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:05.547677 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:05.547677 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:05.547677 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:05.547677 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:05.547677 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:05.547677 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:05.548614 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:05.548614 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:05.548614 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:05.548614 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:05.548614 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:05.548614 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:05.548614 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:05.548614 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:05.548614 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:05.548614 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:05.548614 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:05.548614 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:05.548614 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:05.548614 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:05.548614 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:05.548614 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:05.550169 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:05.550169 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:05.550169 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:05.550169 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:05.550169 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:05.550169 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:05.550169 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:05.550169 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:05.550169 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:05.550169 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:05.550169 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:40:05.550169 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:05.553024 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 160/327] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:40:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:05.558848 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86de04c-00e9-40d5-819d-809a83cb453a None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:05.674081 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32834862-1f2a-4a40-b3be-8a8dfa2486d6 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:05.697389 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86de04c-00e9-40d5-819d-809a83cb453a None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:05.700706 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:05.717767 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f806e-b105-4102-84f5-dc661bad1acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:05.718586 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f806e-b105-4102-84f5-dc661bad1acf None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:05.718877 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f806e-b105-4102-84f5-dc661bad1acf None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:05.719155 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f806e-b105-4102-84f5-dc661bad1acf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:05.721346 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:05.730833 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292f806e-b105-4102-84f5-dc661bad1acf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:05.738809 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-292f806e-b105-4102-84f5-dc661bad1acf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 17 16:40:05.738809 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:05.738809 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:05.738809 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:05.738809 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:05.738809 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:05.738809 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:05.738809 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:05.738809 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:05.738809 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:05.738809 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:05.738809 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:05.738809 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:05.738809 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:05.738809 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:05.738809 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:05.739912 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:05.739912 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:05.739912 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:05.739912 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:05.739912 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:05.739912 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:05.739912 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:05.739912 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:05.739912 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:05.739912 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:05.739912 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:05.739912 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:05.739912 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:05.739912 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:05.739912 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:05.739912 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:05.740911 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:05.740911 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:05.740911 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:05.740911 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:05.740911 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:05.740911 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:05.740911 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:05.740911 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:05.740911 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:05.740911 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:05.740911 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 17 16:40:05.740911 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:05.740911 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 168/328] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:40:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:05.743071 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987faaa6-5457-40ae-9f21-5b66695abaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:05.743918 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987faaa6-5457-40ae-9f21-5b66695abaf4 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:05.744337 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987faaa6-5457-40ae-9f21-5b66695abaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:05.744755 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987faaa6-5457-40ae-9f21-5b66695abaf4 None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:05.745709 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987faaa6-5457-40ae-9f21-5b66695abaf4 None admin] RBAC: Adding query filter params (name=member) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:05.745871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f806e-b105-4102-84f5-dc661bad1acf None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:05.746290 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987faaa6-5457-40ae-9f21-5b66695abaf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:05.747897 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987faaa6-5457-40ae-9f21-5b66695abaf4 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:05.763113 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 161/329] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 17 16:40:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:05.769773 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987faaa6-5457-40ae-9f21-5b66695abaf4 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:05.869354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f806e-b105-4102-84f5-dc661bad1acf None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:05.898030 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987faaa6-5457-40ae-9f21-5b66695abaf4 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:05.918561 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6314bed-b40f-4660-a18b-2d6928188038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:05.919257 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6314bed-b40f-4660-a18b-2d6928188038 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:05.919712 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6314bed-b40f-4660-a18b-2d6928188038 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:05.919999 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6314bed-b40f-4660-a18b-2d6928188038 None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:05.921101 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6314bed-b40f-4660-a18b-2d6928188038 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:05.921418 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6314bed-b40f-4660-a18b-2d6928188038 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:05.922833 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6314bed-b40f-4660-a18b-2d6928188038 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:05.935460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 169/330] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 17 16:40:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:40:05.941648 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6314bed-b40f-4660-a18b-2d6928188038 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:05.941936 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde6e80f-7b4c-4e1b-ac15-c0fc5664678e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:05.942883 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6e80f-7b4c-4e1b-ac15-c0fc5664678e None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:05.943125 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6e80f-7b4c-4e1b-ac15-c0fc5664678e None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:05.943404 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6e80f-7b4c-4e1b-ac15-c0fc5664678e None admin] PATH_INFO: `/v3/users/180f5f3585c241a2993e43d06b130d7d` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:05.945044 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde6e80f-7b4c-4e1b-ac15-c0fc5664678e None admin] RBAC: Authorizing `identity:get_user(user_id=180f5f3585c241a2993e43d06b130d7d)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:05.947810 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde6e80f-7b4c-4e1b-ac15-c0fc5664678e None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:05.949830 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 162/331] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:40:05 2022] GET /identity/v3/users/180f5f3585c241a2993e43d06b130d7d => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:40:05.959363 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde6e80f-7b4c-4e1b-ac15-c0fc5664678e None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:06.012396 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:06.029283 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:06.062025 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6314bed-b40f-4660-a18b-2d6928188038 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:06.081734 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde6e80f-7b4c-4e1b-ac15-c0fc5664678e None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:06.103313 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6416529b-924c-465f-b092-ab19764a1342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:06.104204 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416529b-924c-465f-b092-ab19764a1342 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:06.104496 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416529b-924c-465f-b092-ab19764a1342 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:06.104773 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416529b-924c-465f-b092-ab19764a1342 None admin] PATH_INFO: `/v3/users/8e00631bc0db4adaba3c76db98febe9e` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:06.106057 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6416529b-924c-465f-b092-ab19764a1342 None admin] RBAC: Authorizing `identity:get_user(user_id=8e00631bc0db4adaba3c76db98febe9e)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:06.107525 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6416529b-924c-465f-b092-ab19764a1342 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:06.109611 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 170/332] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:40:05 2022] GET /identity/v3/users/8e00631bc0db4adaba3c76db98febe9e => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:06.114878 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6416529b-924c-465f-b092-ab19764a1342 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:06.157541 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0533e62a-ffb5-43ad-af66-9973e91f635b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:06.158374 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533e62a-ffb5-43ad-af66-9973e91f635b None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:06.159344 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533e62a-ffb5-43ad-af66-9973e91f635b None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:06.159703 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533e62a-ffb5-43ad-af66-9973e91f635b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:06.160639 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0533e62a-ffb5-43ad-af66-9973e91f635b None admin] RBAC: Adding query filter params (role.id=a47f5df8505f49419d0acb4f0f287610, user.id=180f5f3585c241a2993e43d06b130d7d, scope.system=all) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:06.161062 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0533e62a-ffb5-43ad-af66-9973e91f635b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:06.162641 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0533e62a-ffb5-43ad-af66-9973e91f635b None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:06.206794 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 163/333] 10.222.0.40 () {56 vars in 1308 bytes} [Sun Jul 17 16:40:05 2022] GET /identity/v3/role_assignments?user.id=180f5f3585c241a2993e43d06b130d7d&scope.system=all&role.id=a47f5df8505f49419d0acb4f0f287610 => generated 510 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:06.250987 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6416529b-924c-465f-b092-ab19764a1342 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:06.279578 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:06.297223 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f8080-2820-4b66-a213-0e19e4b0e47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:06.298397 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f8080-2820-4b66-a213-0e19e4b0e47a None admin] REQUEST_METHOD: `PUT` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:06.298854 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f8080-2820-4b66-a213-0e19e4b0e47a None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:06.299877 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f8080-2820-4b66-a213-0e19e4b0e47a None admin] PATH_INFO: `/v3/system/users/8e00631bc0db4adaba3c76db98febe9e/roles/e7249313e5334765ba16793745caed63` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:06.305769 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f8080-2820-4b66-a213-0e19e4b0e47a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8e00631bc0db4adaba3c76db98febe9e, role_id=e7249313e5334765ba16793745caed63)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:06.307576 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f8080-2820-4b66-a213-0e19e4b0e47a None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:06.379104 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 171/334] 10.222.0.40 () {58 vars in 1334 bytes} [Sun Jul 17 16:40:06 2022] PUT /identity/v3/system/users/8e00631bc0db4adaba3c76db98febe9e/roles/e7249313e5334765ba16793745caed63 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 17 16:40:07.951546 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a48b2-d1b1-4b60-a91c-53d3f8db7392 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:07.952059 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a48b2-d1b1-4b60-a91c-53d3f8db7392 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:07.952134 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a48b2-d1b1-4b60-a91c-53d3f8db7392 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:07.953554 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 164/335] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:07.959220 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6855ae02-7392-4f63-8d3d-24c51cbd5839 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:07.960194 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6855ae02-7392-4f63-8d3d-24c51cbd5839 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:07.960694 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6855ae02-7392-4f63-8d3d-24c51cbd5839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:07.990306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-6855ae02-7392-4f63-8d3d-24c51cbd5839 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:07.992765 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-6855ae02-7392-4f63-8d3d-24c51cbd5839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:08.111281 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 172/336] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:08.152989 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a48b2-d1b1-4b60-a91c-53d3f8db7392 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:08.277608 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:08.294329 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a48b2-d1b1-4b60-a91c-53d3f8db7392 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:08.303023 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bfd5d-e58e-4e97-9ddc-b22b8aff9dd1 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:08.303668 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bfd5d-e58e-4e97-9ddc-b22b8aff9dd1 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:08.304187 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bfd5d-e58e-4e97-9ddc-b22b8aff9dd1 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:08.306302 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 173/337] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:08.312015 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb803cec-3008-4f12-90e3-800bf07c9c9a None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:08.312646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb803cec-3008-4f12-90e3-800bf07c9c9a None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:08.313237 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb803cec-3008-4f12-90e3-800bf07c9c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:08.344633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72173f45-18b9-41b1-abc1-d07fb03dfc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:08.345756 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72173f45-18b9-41b1-abc1-d07fb03dfc9f None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:08.346742 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72173f45-18b9-41b1-abc1-d07fb03dfc9f None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:08.347170 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72173f45-18b9-41b1-abc1-d07fb03dfc9f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:08.360832 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72173f45-18b9-41b1-abc1-d07fb03dfc9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:08.362337 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-cb803cec-3008-4f12-90e3-800bf07c9c9a None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:08.364018 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-cb803cec-3008-4f12-90e3-800bf07c9c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:08.371984 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-72173f45-18b9-41b1-abc1-d07fb03dfc9f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 17 16:40:08.371984 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:08.371984 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:08.371984 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:08.371984 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:08.371984 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:08.371984 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:08.371984 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:08.371984 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:08.371984 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:08.371984 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:08.371984 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:08.371984 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:08.371984 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:08.371984 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:08.371984 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:08.373007 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:08.373007 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:08.373007 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:08.373007 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:08.373007 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:08.373007 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:08.373007 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:08.373007 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:08.373007 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:08.373007 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:08.373007 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:08.373007 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:08.373007 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:08.373007 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:08.373007 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:08.373007 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:08.373809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:08.373809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:08.373809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:08.373809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:08.373809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:08.373809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:08.373809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:08.373809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:08.373809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:08.373809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:08.373809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 17 16:40:08.373809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:08.376757 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 165/338] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 17 16:40:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:08.382795 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72173f45-18b9-41b1-abc1-d07fb03dfc9f None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:08.539048 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72173f45-18b9-41b1-abc1-d07fb03dfc9f None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:08.547615 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 174/339] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:08.586910 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca8057-98e7-4ea3-a9a1-d5f4e5ab77ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:08.587614 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca8057-98e7-4ea3-a9a1-d5f4e5ab77ac None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:08.593290 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca8057-98e7-4ea3-a9a1-d5f4e5ab77ac None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:08.593290 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca8057-98e7-4ea3-a9a1-d5f4e5ab77ac None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:08.593290 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ca8057-98e7-4ea3-a9a1-d5f4e5ab77ac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:08.593290 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ca8057-98e7-4ea3-a9a1-d5f4e5ab77ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:08.593290 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ca8057-98e7-4ea3-a9a1-d5f4e5ab77ac None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:08.604845 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 166/340] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:40:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:40:08.607693 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb803cec-3008-4f12-90e3-800bf07c9c9a None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:08.610408 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca8057-98e7-4ea3-a9a1-d5f4e5ab77ac None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:08.637212 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:09.050747 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:09.128822 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca8057-98e7-4ea3-a9a1-d5f4e5ab77ac None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:09.175509 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2280e12-e928-4e56-818d-6612a98ad518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:09.176201 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2280e12-e928-4e56-818d-6612a98ad518 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:09.176492 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2280e12-e928-4e56-818d-6612a98ad518 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:09.176796 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2280e12-e928-4e56-818d-6612a98ad518 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:09.179009 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb803cec-3008-4f12-90e3-800bf07c9c9a None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:09.238927 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edbdbcb-a4f3-45a8-8b57-ed7a703b4300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:09.239763 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbdbcb-a4f3-45a8-8b57-ed7a703b4300 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:09.240052 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbdbcb-a4f3-45a8-8b57-ed7a703b4300 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:09.240332 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbdbcb-a4f3-45a8-8b57-ed7a703b4300 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:09.260807 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2280e12-e928-4e56-818d-6612a98ad518 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:09.261021 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edbdbcb-a4f3-45a8-8b57-ed7a703b4300 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:09.262577 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2280e12-e928-4e56-818d-6612a98ad518 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:09.277306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-9edbdbcb-a4f3-45a8-8b57-ed7a703b4300 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:40:09.277306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:09.277306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:09.277306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:09.277306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:09.277306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:09.277306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:09.277306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:09.277306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:09.277306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:09.277306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:09.277306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:09.277306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:09.277306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:09.277306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:09.277306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:09.278315 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:09.278315 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:09.278315 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:09.278315 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:09.278315 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:09.278315 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:09.278315 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:09.278315 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:09.278315 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:09.278315 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:09.278315 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:09.278315 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:09.278315 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:09.278315 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:09.278315 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:09.278315 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:09.279571 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:09.279571 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:09.279571 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:09.279571 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:09.279571 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:09.279571 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:09.279571 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:09.279571 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:09.279571 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:09.279571 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:09.279571 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:40:09.279571 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:09.279571 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 175/341] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:40:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 672 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:40:09.280283 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95dc02-2012-4c63-8c9c-8cc8e6b22d11 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:09.280584 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95dc02-2012-4c63-8c9c-8cc8e6b22d11 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:09.280922 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95dc02-2012-4c63-8c9c-8cc8e6b22d11 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:09.282579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 176/342] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:09.283129 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:09.290455 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223c4cc-b37c-4d41-aef3-e51fcee86196 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:09.290777 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223c4cc-b37c-4d41-aef3-e51fcee86196 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:09.291026 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223c4cc-b37c-4d41-aef3-e51fcee86196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:09.330009 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-1223c4cc-b37c-4d41-aef3-e51fcee86196 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:09.331286 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-1223c4cc-b37c-4d41-aef3-e51fcee86196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:09.383637 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 167/343] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:40:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 774 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 17 16:40:09.384538 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2280e12-e928-4e56-818d-6612a98ad518 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:09.447354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 177/344] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:09.448261 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1223c4cc-b37c-4d41-aef3-e51fcee86196 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:09.516930 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:09.527079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2280e12-e928-4e56-818d-6612a98ad518 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:09.544320 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:09.568986 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4019848-c962-4652-b6d8-43db8f312863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:09.569999 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4019848-c962-4652-b6d8-43db8f312863 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:09.570444 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4019848-c962-4652-b6d8-43db8f312863 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:09.571127 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4019848-c962-4652-b6d8-43db8f312863 None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:09.572072 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4019848-c962-4652-b6d8-43db8f312863 None admin] RBAC: Adding query filter params (name=member) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:09.572548 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4019848-c962-4652-b6d8-43db8f312863 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:09.574254 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4019848-c962-4652-b6d8-43db8f312863 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:09.585467 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 168/345] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 17 16:40:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:09.586629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4019848-c962-4652-b6d8-43db8f312863 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:09.652478 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1223c4cc-b37c-4d41-aef3-e51fcee86196 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:09.704030 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4019848-c962-4652-b6d8-43db8f312863 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:09.721830 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab84a33-fe01-4707-afff-7efaeaecad81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:09.723613 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab84a33-fe01-4707-afff-7efaeaecad81 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:09.724110 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab84a33-fe01-4707-afff-7efaeaecad81 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:09.724744 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab84a33-fe01-4707-afff-7efaeaecad81 None admin] PATH_INFO: `/v3/projects/fde6b97a4d09416a81257067f1d1b919` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:09.726930 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab84a33-fe01-4707-afff-7efaeaecad81 None admin] RBAC: Authorizing `identity:get_project(project_id=fde6b97a4d09416a81257067f1d1b919)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:09.731207 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab84a33-fe01-4707-afff-7efaeaecad81 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:09.735141 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 178/346] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:40:09 2022] GET /identity/v3/projects/fde6b97a4d09416a81257067f1d1b919 => generated 298 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:09.743719 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c208b6-2648-4985-bc1f-8eeceb77ab18 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:09.744619 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c208b6-2648-4985-bc1f-8eeceb77ab18 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:09.745335 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c208b6-2648-4985-bc1f-8eeceb77ab18 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:09.746829 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de719a-015a-466e-956f-418c2519390f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:09.747832 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de719a-015a-466e-956f-418c2519390f None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:09.748025 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 179/347] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:09 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:09.749300 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c208b6-2648-4985-bc1f-8eeceb77ab18 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:09.749424 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de719a-015a-466e-956f-418c2519390f None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:09.749666 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de719a-015a-466e-956f-418c2519390f None admin] PATH_INFO: `/v3/roles/member` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:09.762314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08de719a-015a-466e-956f-418c2519390f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:09.773346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-08de719a-015a-466e-956f-418c2519390f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:40:09.773346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:09.773346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:09.773346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:09.773346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:09.773346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:09.773346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:09.773346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:09.773346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:09.773346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:09.773346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:09.773346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:09.773346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:09.773346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:09.773346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:09.773346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:09.785267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:09.785267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:09.785267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:09.785267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:09.785267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:09.785267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:09.785267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:09.785267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:09.785267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:09.785267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:09.785267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:09.785267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:09.785267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:09.785267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:09.785267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:09.785267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:09.786079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:09.786079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:09.786079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:09.786079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:09.786079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:09.786079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:09.786079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:09.786079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:09.786079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:09.786079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:09.786079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:40:09.786079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:09.786079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 169/348] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:40:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:09.786079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232144f9-c7ef-4f53-9d69-7b8f8e36ae07 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:09.786824 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232144f9-c7ef-4f53-9d69-7b8f8e36ae07 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:09.786824 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232144f9-c7ef-4f53-9d69-7b8f8e36ae07 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:09.786824 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 170/349] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:09.786824 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232144f9-c7ef-4f53-9d69-7b8f8e36ae07 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:09.828094 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:09.936230 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232144f9-c7ef-4f53-9d69-7b8f8e36ae07 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:09.944358 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c208b6-2648-4985-bc1f-8eeceb77ab18 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:09.954511 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:09.993614 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567d646-c36b-4406-bbdf-6c034e3cc16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:09.994288 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567d646-c36b-4406-bbdf-6c034e3cc16a None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:09.994580 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567d646-c36b-4406-bbdf-6c034e3cc16a None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:09.994896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567d646-c36b-4406-bbdf-6c034e3cc16a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:09.996813 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ef3260-911b-4c7c-84a5-20af2a1f3283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:09.998019 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef3260-911b-4c7c-84a5-20af2a1f3283 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:09.998424 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef3260-911b-4c7c-84a5-20af2a1f3283 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:09.998867 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ef3260-911b-4c7c-84a5-20af2a1f3283 None admin] PATH_INFO: `/v3/groups/490d3612ce324804bb32d642c6b92f03` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:10.011196 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ef3260-911b-4c7c-84a5-20af2a1f3283 None admin] RBAC: Authorizing `identity:get_group(group_id=490d3612ce324804bb32d642c6b92f03)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:10.013928 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ef3260-911b-4c7c-84a5-20af2a1f3283 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:10.016531 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 171/350] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Jul 17 16:40:09 2022] GET /identity/v3/groups/490d3612ce324804bb32d642c6b92f03 => generated 226 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:10.019723 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba97185-d7d8-4fe1-b593-baf4fa8ef1d5 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:10.020175 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba97185-d7d8-4fe1-b593-baf4fa8ef1d5 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:10.020595 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba97185-d7d8-4fe1-b593-baf4fa8ef1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:10.045745 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-8ba97185-d7d8-4fe1-b593-baf4fa8ef1d5 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:10.047377 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-8ba97185-d7d8-4fe1-b593-baf4fa8ef1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:10.106705 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567d646-c36b-4406-bbdf-6c034e3cc16a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:10.109578 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567d646-c36b-4406-bbdf-6c034e3cc16a None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:10.142349 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 172/351] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:10.144444 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba97185-d7d8-4fe1-b593-baf4fa8ef1d5 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:10.194666 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:10.255496 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 180/352] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:40:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 17 16:40:10.257204 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e024325-f1dd-43ee-8f7c-8067c5f2a08e None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:10.257680 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e024325-f1dd-43ee-8f7c-8067c5f2a08e None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:10.258119 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e024325-f1dd-43ee-8f7c-8067c5f2a08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:10.274753 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:10.279172 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-5e024325-f1dd-43ee-8f7c-8067c5f2a08e None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:10.280422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-5e024325-f1dd-43ee-8f7c-8067c5f2a08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:10.314169 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba97185-d7d8-4fe1-b593-baf4fa8ef1d5 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:10.356906 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 181/353] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:10.358117 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e024325-f1dd-43ee-8f7c-8067c5f2a08e None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:10.368542 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6548e5f3-c0cb-4d60-8168-5ce7ccb16755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:10.369562 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6548e5f3-c0cb-4d60-8168-5ce7ccb16755 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:10.369954 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6548e5f3-c0cb-4d60-8168-5ce7ccb16755 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:10.370332 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6548e5f3-c0cb-4d60-8168-5ce7ccb16755 None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:10.371471 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6548e5f3-c0cb-4d60-8168-5ce7ccb16755 None admin] RBAC: Adding query filter params (name=member) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:10.371690 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6548e5f3-c0cb-4d60-8168-5ce7ccb16755 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:10.373264 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6548e5f3-c0cb-4d60-8168-5ce7ccb16755 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:10.384932 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 173/354] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 17 16:40:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:10.385946 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6548e5f3-c0cb-4d60-8168-5ce7ccb16755 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:10.483686 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e024325-f1dd-43ee-8f7c-8067c5f2a08e None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:10.514351 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:10.527305 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6548e5f3-c0cb-4d60-8168-5ce7ccb16755 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:10.531959 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57145085-3a70-418e-a4e9-4c9210a35d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:10.531959 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57145085-3a70-418e-a4e9-4c9210a35d77 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:10.531959 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57145085-3a70-418e-a4e9-4c9210a35d77 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:10.531959 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57145085-3a70-418e-a4e9-4c9210a35d77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:10.537427 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57145085-3a70-418e-a4e9-4c9210a35d77 None admin] RBAC: Adding query filter params (group.id=490d3612ce324804bb32d642c6b92f03, role.id=71cf67eb04704663aff719f6e3804f1a, scope.project.id=fde6b97a4d09416a81257067f1d1b919) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:10.537427 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57145085-3a70-418e-a4e9-4c9210a35d77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:10.537427 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57145085-3a70-418e-a4e9-4c9210a35d77 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:10.537427 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:10.539699 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 182/355] 10.222.0.40 () {56 vars in 1376 bytes} [Sun Jul 17 16:40:10 2022] GET /identity/v3/role_assignments?group.id=490d3612ce324804bb32d642c6b92f03&scope.project.id=fde6b97a4d09416a81257067f1d1b919&role.id=71cf67eb04704663aff719f6e3804f1a => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:10.540589 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57145085-3a70-418e-a4e9-4c9210a35d77 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:10.608885 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d0976-44f9-4ea3-8661-e114b781c1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:10.609786 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d0976-44f9-4ea3-8661-e114b781c1a6 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:10.610061 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d0976-44f9-4ea3-8661-e114b781c1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:10.610355 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3d0976-44f9-4ea3-8661-e114b781c1a6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:10.624848 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3d0976-44f9-4ea3-8661-e114b781c1a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:10.635921 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-7d3d0976-44f9-4ea3-8661-e114b781c1a6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 17 16:40:10.635921 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:10.635921 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:10.635921 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:10.635921 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:10.635921 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:10.635921 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:10.635921 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:10.635921 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:10.635921 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:10.635921 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:10.635921 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:10.635921 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:10.635921 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:10.635921 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:10.635921 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:10.637043 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:10.637043 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:10.637043 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:10.637043 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:10.637043 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:10.637043 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:10.637043 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:10.637043 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:10.637043 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:10.637043 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:10.637043 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:10.637043 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:10.637043 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:10.637043 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:10.637043 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:10.637043 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:10.637856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:10.637856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:10.637856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:10.637856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:10.637856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:10.637856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:10.637856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:10.637856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:10.637856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:10.637856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:10.637856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 17 16:40:10.637856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:10.637856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 174/356] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:40:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:10.639672 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d0976-44f9-4ea3-8661-e114b781c1a6 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:10.677165 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57145085-3a70-418e-a4e9-4c9210a35d77 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:10.722197 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28e5b0-89ad-4396-becf-2712077d3ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:10.723305 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28e5b0-89ad-4396-becf-2712077d3ec9 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:10.723598 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28e5b0-89ad-4396-becf-2712077d3ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:10.723885 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28e5b0-89ad-4396-becf-2712077d3ec9 None admin] PATH_INFO: `/v3/projects/fde6b97a4d09416a81257067f1d1b919` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:10.725326 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28e5b0-89ad-4396-becf-2712077d3ec9 None admin] RBAC: Authorizing `identity:get_project(project_id=fde6b97a4d09416a81257067f1d1b919)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:10.726853 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28e5b0-89ad-4396-becf-2712077d3ec9 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:10.729174 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 183/357] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:40:10 2022] GET /identity/v3/projects/fde6b97a4d09416a81257067f1d1b919 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:10.730253 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28e5b0-89ad-4396-becf-2712077d3ec9 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:10.777169 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3d0976-44f9-4ea3-8661-e114b781c1a6 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:10.812892 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:10.839811 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b24a416-4fc0-4707-a303-dc8f6b85e109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:10.841573 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24a416-4fc0-4707-a303-dc8f6b85e109 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:10.841760 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24a416-4fc0-4707-a303-dc8f6b85e109 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:10.842263 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24a416-4fc0-4707-a303-dc8f6b85e109 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:10.865118 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b24a416-4fc0-4707-a303-dc8f6b85e109 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:10.873197 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28e5b0-89ad-4396-becf-2712077d3ec9 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:10.874101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-1b24a416-4fc0-4707-a303-dc8f6b85e109 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 17 16:40:10.874101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:10.874101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:10.874101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:10.874101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:10.874101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:10.874101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:10.874101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:10.874101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:10.874101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:10.874101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:10.874101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:10.874101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:10.874101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:10.874101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:10.874101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:10.874957 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:10.874957 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:10.874957 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:10.874957 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:10.874957 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:10.874957 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:10.874957 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:10.874957 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:10.874957 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:10.874957 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:10.874957 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:10.874957 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:10.874957 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:10.874957 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:10.874957 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:10.874957 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:10.875906 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:10.875906 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:10.875906 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:10.875906 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:10.875906 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:10.875906 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:10.875906 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:10.875906 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:10.875906 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:10.875906 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:10.875906 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 17 16:40:10.875906 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:10.875906 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 175/358] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 17 16:40:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:10.877738 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e5ab2-0f97-4269-87df-bb2bb9b34a87 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:10.878035 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e5ab2-0f97-4269-87df-bb2bb9b34a87 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:10.878319 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e5ab2-0f97-4269-87df-bb2bb9b34a87 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:10.879936 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 176/359] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:10.880709 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e5ab2-0f97-4269-87df-bb2bb9b34a87 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:10.903355 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:10.934150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a9ccb-5a12-406b-9507-6d84522d76a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:10.935195 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a9ccb-5a12-406b-9507-6d84522d76a9 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:10.935715 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a9ccb-5a12-406b-9507-6d84522d76a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:10.936174 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a9ccb-5a12-406b-9507-6d84522d76a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:11.051379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a9ccb-5a12-406b-9507-6d84522d76a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:11.054473 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a9ccb-5a12-406b-9507-6d84522d76a9 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:11.054885 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e5ab2-0f97-4269-87df-bb2bb9b34a87 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:11.100254 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b806f-a3e7-4d89-97e3-9ce5bd570d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:11.101001 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b806f-a3e7-4d89-97e3-9ce5bd570d6c None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:11.101289 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b806f-a3e7-4d89-97e3-9ce5bd570d6c None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:11.101573 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b806f-a3e7-4d89-97e3-9ce5bd570d6c None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:11.102405 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4b806f-a3e7-4d89-97e3-9ce5bd570d6c None admin] RBAC: Adding query filter params (name=reader) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:11.102722 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4b806f-a3e7-4d89-97e3-9ce5bd570d6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:11.104875 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4b806f-a3e7-4d89-97e3-9ce5bd570d6c None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:11.116008 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 177/360] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 17 16:40:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:11.117022 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b806f-a3e7-4d89-97e3-9ce5bd570d6c None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:11.150412 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:11.202743 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 184/361] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:40:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 17 16:40:11.203741 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a9ccb-5a12-406b-9507-6d84522d76a9 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:11.230649 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:11.270438 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b806f-a3e7-4d89-97e3-9ce5bd570d6c None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:11.313485 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb70726-5ff3-4f38-bc98-d5e974dae212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:11.314347 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb70726-5ff3-4f38-bc98-d5e974dae212 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:11.315314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb70726-5ff3-4f38-bc98-d5e974dae212 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:11.315930 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb70726-5ff3-4f38-bc98-d5e974dae212 None admin] PATH_INFO: `/v3/groups/490d3612ce324804bb32d642c6b92f03` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:11.318068 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb70726-5ff3-4f38-bc98-d5e974dae212 None admin] RBAC: Authorizing `identity:get_group(group_id=490d3612ce324804bb32d642c6b92f03)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:11.318460 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb70726-5ff3-4f38-bc98-d5e974dae212 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:11.320166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 178/362] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Jul 17 16:40:11 2022] GET /identity/v3/groups/490d3612ce324804bb32d642c6b92f03 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:11.324938 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d08a4c-5c07-4c79-b625-d361c5e5732f None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:11.324938 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d08a4c-5c07-4c79-b625-d361c5e5732f None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:11.324938 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d08a4c-5c07-4c79-b625-d361c5e5732f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:11.344070 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-c9d08a4c-5c07-4c79-b625-d361c5e5732f None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:11.345627 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-c9d08a4c-5c07-4c79-b625-d361c5e5732f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:11.349759 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a9ccb-5a12-406b-9507-6d84522d76a9 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:11.392759 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e57201-045c-4f47-9154-c21c6173c500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:11.393602 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e57201-045c-4f47-9154-c21c6173c500 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:11.394120 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e57201-045c-4f47-9154-c21c6173c500 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:11.394477 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e57201-045c-4f47-9154-c21c6173c500 None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:11.395375 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e57201-045c-4f47-9154-c21c6173c500 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:11.395579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e57201-045c-4f47-9154-c21c6173c500 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:11.397919 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e57201-045c-4f47-9154-c21c6173c500 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:11.410383 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 185/363] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:40:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:11.410383 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e57201-045c-4f47-9154-c21c6173c500 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:11.437437 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 179/364] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:11.438177 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d08a4c-5c07-4c79-b625-d361c5e5732f None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:11.450961 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:11.538906 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e57201-045c-4f47-9154-c21c6173c500 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:11.540222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:11.581710 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d08a4c-5c07-4c79-b625-d361c5e5732f None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:11.592222 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d13d035-c673-40a9-91df-6fde6515b540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:11.593439 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13d035-c673-40a9-91df-6fde6515b540 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:11.593915 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13d035-c673-40a9-91df-6fde6515b540 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:11.594203 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13d035-c673-40a9-91df-6fde6515b540 None admin] PATH_INFO: `/v3/users/8e00631bc0db4adaba3c76db98febe9e` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:11.595527 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d13d035-c673-40a9-91df-6fde6515b540 None admin] RBAC: Authorizing `identity:get_user(user_id=8e00631bc0db4adaba3c76db98febe9e)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:11.598056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d13d035-c673-40a9-91df-6fde6515b540 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:11.600005 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 186/365] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:40:11 2022] GET /identity/v3/users/8e00631bc0db4adaba3c76db98febe9e => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:11.600640 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d13d035-c673-40a9-91df-6fde6515b540 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:11.621823 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98ad5ff-6048-4915-8e20-8429983c015e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:11.623005 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ad5ff-6048-4915-8e20-8429983c015e None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:11.623005 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ad5ff-6048-4915-8e20-8429983c015e None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:11.623277 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ad5ff-6048-4915-8e20-8429983c015e None admin] PATH_INFO: `/v3/projects/6704fc6d36e84a078282ac2057784505` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:11.624569 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98ad5ff-6048-4915-8e20-8429983c015e None admin] RBAC: Authorizing `identity:get_project(project_id=6704fc6d36e84a078282ac2057784505)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:11.626095 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98ad5ff-6048-4915-8e20-8429983c015e None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:11.628509 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 180/366] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:40:11 2022] GET /identity/v3/projects/6704fc6d36e84a078282ac2057784505 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:11.629452 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98ad5ff-6048-4915-8e20-8429983c015e None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:11.745000 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d13d035-c673-40a9-91df-6fde6515b540 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:11.747677 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:11.786181 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98ad5ff-6048-4915-8e20-8429983c015e None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:11.787245 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c84dd-f33d-4e7d-bae2-b47197a2d969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:11.787919 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c84dd-f33d-4e7d-bae2-b47197a2d969 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:11.788208 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c84dd-f33d-4e7d-bae2-b47197a2d969 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:11.788493 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8c84dd-f33d-4e7d-bae2-b47197a2d969 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:11.789957 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8c84dd-f33d-4e7d-bae2-b47197a2d969 None admin] RBAC: Adding query filter params (group.id=490d3612ce324804bb32d642c6b92f03, role.id=a47f5df8505f49419d0acb4f0f287610, scope.project.id=fde6b97a4d09416a81257067f1d1b919) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:11.789957 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8c84dd-f33d-4e7d-bae2-b47197a2d969 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:11.791125 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8c84dd-f33d-4e7d-bae2-b47197a2d969 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:11.805336 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 187/367] 10.222.0.40 () {56 vars in 1376 bytes} [Sun Jul 17 16:40:11 2022] GET /identity/v3/role_assignments?group.id=490d3612ce324804bb32d642c6b92f03&scope.project.id=fde6b97a4d09416a81257067f1d1b919&role.id=a47f5df8505f49419d0acb4f0f287610 => generated 260 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:11.806300 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c84dd-f33d-4e7d-bae2-b47197a2d969 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:11.833017 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f7a5d-f9a0-421f-928f-ca57bb8216de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:11.833653 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f7a5d-f9a0-421f-928f-ca57bb8216de None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:11.833956 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f7a5d-f9a0-421f-928f-ca57bb8216de None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:11.834239 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f7a5d-f9a0-421f-928f-ca57bb8216de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:11.850200 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:11.932239 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f7a5d-f9a0-421f-928f-ca57bb8216de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:11.933946 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f7a5d-f9a0-421f-928f-ca57bb8216de None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:12.062193 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8c84dd-f33d-4e7d-bae2-b47197a2d969 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:12.108845 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ed96c1-221d-4366-ab52-c1b04fe9e90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:12.109763 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed96c1-221d-4366-ab52-c1b04fe9e90c None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:12.110054 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed96c1-221d-4366-ab52-c1b04fe9e90c None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:12.110338 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed96c1-221d-4366-ab52-c1b04fe9e90c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:12.122069 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 181/368] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:40:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 17 16:40:12.124043 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f7a5d-f9a0-421f-928f-ca57bb8216de None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:12.129350 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ed96c1-221d-4366-ab52-c1b04fe9e90c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:12.137423 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-05ed96c1-221d-4366-ab52-c1b04fe9e90c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:40:12.137423 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:12.137423 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:12.137423 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:12.137423 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:12.137423 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:12.137423 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:12.137423 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:12.137423 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:12.137423 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:12.137423 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:12.137423 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:12.137423 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:12.137423 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:12.137423 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:12.137423 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:12.139135 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:12.139135 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:12.139135 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:12.139135 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:12.139135 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:12.139135 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:12.139135 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:12.139135 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:12.139135 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:12.139135 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:12.139135 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:12.139135 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:12.139135 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:12.139135 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:12.139135 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:12.139135 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:12.140675 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:12.140675 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:12.140675 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:12.140675 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:12.140675 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:12.140675 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:12.140675 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:12.140675 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:12.140675 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:12.140675 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:12.140675 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:40:12.140675 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:12.144505 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 188/369] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:40:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:12.145608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ed96c1-221d-4366-ab52-c1b04fe9e90c None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:12.197578 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:12.213533 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:12.294759 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f7a5d-f9a0-421f-928f-ca57bb8216de None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:12.335537 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4559e1d-649a-40a1-88c3-f50dc8d9796f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:12.336216 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4559e1d-649a-40a1-88c3-f50dc8d9796f None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:12.336505 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4559e1d-649a-40a1-88c3-f50dc8d9796f None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:12.336812 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4559e1d-649a-40a1-88c3-f50dc8d9796f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:12.337676 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4559e1d-649a-40a1-88c3-f50dc8d9796f None admin] RBAC: Adding query filter params (role.id=e7249313e5334765ba16793745caed63, user.id=8e00631bc0db4adaba3c76db98febe9e, scope.system=all) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:12.337997 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4559e1d-649a-40a1-88c3-f50dc8d9796f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:12.339617 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4559e1d-649a-40a1-88c3-f50dc8d9796f None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:12.361079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 182/370] 10.222.0.40 () {56 vars in 1308 bytes} [Sun Jul 17 16:40:12 2022] GET /identity/v3/role_assignments?user.id=8e00631bc0db4adaba3c76db98febe9e&scope.system=all&role.id=e7249313e5334765ba16793745caed63 => generated 510 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:12.361079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4559e1d-649a-40a1-88c3-f50dc8d9796f None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:12.385706 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ed96c1-221d-4366-ab52-c1b04fe9e90c None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:12.509828 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e766b5-b1cc-46bf-8b5c-edf366a84349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:12.520820 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e766b5-b1cc-46bf-8b5c-edf366a84349 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:12.521970 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e766b5-b1cc-46bf-8b5c-edf366a84349 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:12.522731 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e766b5-b1cc-46bf-8b5c-edf366a84349 None admin] PATH_INFO: `/v3/groups/490d3612ce324804bb32d642c6b92f03` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:12.525137 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e766b5-b1cc-46bf-8b5c-edf366a84349 None admin] RBAC: Authorizing `identity:get_group(group_id=490d3612ce324804bb32d642c6b92f03)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:12.527959 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e766b5-b1cc-46bf-8b5c-edf366a84349 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:12.534113 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 189/371] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Jul 17 16:40:12 2022] GET /identity/v3/groups/490d3612ce324804bb32d642c6b92f03 => generated 226 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:12.534113 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e766b5-b1cc-46bf-8b5c-edf366a84349 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:12.639856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:12.652591 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4559e1d-649a-40a1-88c3-f50dc8d9796f None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:12.654835 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:12.666833 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e766b5-b1cc-46bf-8b5c-edf366a84349 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:12.730519 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6e52d-d32d-4d94-98cc-0819823cf4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:12.732275 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6e52d-d32d-4d94-98cc-0819823cf4bc None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:12.732772 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6e52d-d32d-4d94-98cc-0819823cf4bc None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:12.733462 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f6e52d-d32d-4d94-98cc-0819823cf4bc None admin] PATH_INFO: `/v3/projects/6704fc6d36e84a078282ac2057784505` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:12.738272 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f6e52d-d32d-4d94-98cc-0819823cf4bc None admin] RBAC: Authorizing `identity:get_project(project_id=6704fc6d36e84a078282ac2057784505)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:12.741926 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f6e52d-d32d-4d94-98cc-0819823cf4bc None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:12.748881 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 183/372] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:40:12 2022] GET /identity/v3/projects/6704fc6d36e84a078282ac2057784505 => generated 302 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:12.750069 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6e52d-d32d-4d94-98cc-0819823cf4bc None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:12.757972 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e4c0c-df4f-4b75-8c45-78f776388ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:12.759375 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e4c0c-df4f-4b75-8c45-78f776388ad2 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:12.759889 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e4c0c-df4f-4b75-8c45-78f776388ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:12.760392 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e4c0c-df4f-4b75-8c45-78f776388ad2 None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:12.761759 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e4c0c-df4f-4b75-8c45-78f776388ad2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:12.763271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e4c0c-df4f-4b75-8c45-78f776388ad2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:12.766841 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e4c0c-df4f-4b75-8c45-78f776388ad2 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:12.779420 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 190/373] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:40:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:12.780663 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e4c0c-df4f-4b75-8c45-78f776388ad2 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:12.903690 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f6e52d-d32d-4d94-98cc-0819823cf4bc None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:12.960177 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf305cc-ae66-4c5e-8a21-ec296e4cdc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:12.961645 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf305cc-ae66-4c5e-8a21-ec296e4cdc56 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:12.961908 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf305cc-ae66-4c5e-8a21-ec296e4cdc56 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:12.962449 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf305cc-ae66-4c5e-8a21-ec296e4cdc56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:12.964107 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf305cc-ae66-4c5e-8a21-ec296e4cdc56 None admin] RBAC: Adding query filter params (group.id=490d3612ce324804bb32d642c6b92f03, role.id=a47f5df8505f49419d0acb4f0f287610, scope.project.id=6704fc6d36e84a078282ac2057784505) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:12.964661 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf305cc-ae66-4c5e-8a21-ec296e4cdc56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:12.969983 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf305cc-ae66-4c5e-8a21-ec296e4cdc56 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:12.981525 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 184/374] 10.222.0.40 () {56 vars in 1376 bytes} [Sun Jul 17 16:40:12 2022] GET /identity/v3/role_assignments?group.id=490d3612ce324804bb32d642c6b92f03&scope.project.id=6704fc6d36e84a078282ac2057784505&role.id=a47f5df8505f49419d0acb4f0f287610 => generated 260 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:12.983095 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf305cc-ae66-4c5e-8a21-ec296e4cdc56 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:12.996089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e4c0c-df4f-4b75-8c45-78f776388ad2 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:13.049902 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341d266b-5500-49c6-9e1d-767398023b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:13.050393 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:13.051778 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d266b-5500-49c6-9e1d-767398023b6c None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:13.052278 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d266b-5500-49c6-9e1d-767398023b6c None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:13.052769 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d266b-5500-49c6-9e1d-767398023b6c None admin] PATH_INFO: `/v3/groups/490d3612ce324804bb32d642c6b92f03` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:13.055275 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341d266b-5500-49c6-9e1d-767398023b6c None admin] RBAC: Authorizing `identity:get_group(group_id=490d3612ce324804bb32d642c6b92f03)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:13.058126 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341d266b-5500-49c6-9e1d-767398023b6c None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:13.061758 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 191/375] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Jul 17 16:40:12 2022] GET /identity/v3/groups/490d3612ce324804bb32d642c6b92f03 => generated 226 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:13.074583 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341d266b-5500-49c6-9e1d-767398023b6c None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:13.168192 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:13.286093 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf305cc-ae66-4c5e-8a21-ec296e4cdc56 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:13.286962 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341d266b-5500-49c6-9e1d-767398023b6c None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:13.332968 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809242e2-e4d4-4d5e-95e8-3c0bd143bacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:13.333640 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809242e2-e4d4-4d5e-95e8-3c0bd143bacc None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:13.333932 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809242e2-e4d4-4d5e-95e8-3c0bd143bacc None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:13.334215 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809242e2-e4d4-4d5e-95e8-3c0bd143bacc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:13.338053 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d116e96-ddaf-489d-8c37-f3855cec7a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:13.339016 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d116e96-ddaf-489d-8c37-f3855cec7a95 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:13.339489 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d116e96-ddaf-489d-8c37-f3855cec7a95 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:13.339951 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d116e96-ddaf-489d-8c37-f3855cec7a95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:13.341447 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d116e96-ddaf-489d-8c37-f3855cec7a95 None admin] RBAC: Adding query filter params (group.id=490d3612ce324804bb32d642c6b92f03, role.id=71cf67eb04704663aff719f6e3804f1a, scope.project.id=6704fc6d36e84a078282ac2057784505) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:13.341981 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d116e96-ddaf-489d-8c37-f3855cec7a95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:13.343667 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d116e96-ddaf-489d-8c37-f3855cec7a95 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:13.354700 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 192/376] 10.222.0.40 () {56 vars in 1376 bytes} [Sun Jul 17 16:40:13 2022] GET /identity/v3/role_assignments?group.id=490d3612ce324804bb32d642c6b92f03&scope.project.id=6704fc6d36e84a078282ac2057784505&role.id=71cf67eb04704663aff719f6e3804f1a => generated 260 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:13.416504 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809242e2-e4d4-4d5e-95e8-3c0bd143bacc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:13.418262 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809242e2-e4d4-4d5e-95e8-3c0bd143bacc None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:13.452819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:13.545070 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 185/377] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:40:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 17 16:40:13.554039 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d116e96-ddaf-489d-8c37-f3855cec7a95 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:13.701060 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:13.721741 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d116e96-ddaf-489d-8c37-f3855cec7a95 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:13.775986 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c6f670-20d4-41ac-8dbc-d4f972e0d78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:13.776935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6f670-20d4-41ac-8dbc-d4f972e0d78a None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:13.777230 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6f670-20d4-41ac-8dbc-d4f972e0d78a None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:13.777512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6f670-20d4-41ac-8dbc-d4f972e0d78a None admin] PATH_INFO: `/v3/projects/3b2ec585f5d6476389a82d18a223a808` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:13.779059 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c6f670-20d4-41ac-8dbc-d4f972e0d78a None admin] RBAC: Authorizing `identity:get_project(project_id=3b2ec585f5d6476389a82d18a223a808)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:13.780655 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c6f670-20d4-41ac-8dbc-d4f972e0d78a None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:13.782531 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51767905-3cfd-4ff9-a847-95a2b1951388 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:13.782714 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 193/378] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:40:13 2022] GET /identity/v3/projects/3b2ec585f5d6476389a82d18a223a808 => generated 344 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:13.784340 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51767905-3cfd-4ff9-a847-95a2b1951388 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:13.788113 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51767905-3cfd-4ff9-a847-95a2b1951388 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:13.788938 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 186/379] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:13.789745 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c6f670-20d4-41ac-8dbc-d4f972e0d78a None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:13.793609 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02f902-7c94-4248-acdf-4e9f907d903c None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:13.793941 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02f902-7c94-4248-acdf-4e9f907d903c None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:13.794187 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02f902-7c94-4248-acdf-4e9f907d903c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:13.829423 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-6b02f902-7c94-4248-acdf-4e9f907d903c None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:13.830876 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-6b02f902-7c94-4248-acdf-4e9f907d903c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:13.977633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 187/380] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 17 16:40:14.006022 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c6f670-20d4-41ac-8dbc-d4f972e0d78a None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:14.041706 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b02f902-7c94-4248-acdf-4e9f907d903c None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:14.068249 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c77cee-654b-4a69-8074-411b7cf5b9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:14.069169 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c77cee-654b-4a69-8074-411b7cf5b9fc None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:14.069459 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c77cee-654b-4a69-8074-411b7cf5b9fc None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:14.069738 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c77cee-654b-4a69-8074-411b7cf5b9fc None admin] PATH_INFO: `/v3/groups/5ce458bca7f34105b7709b4e4447bb29` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:14.083508 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c77cee-654b-4a69-8074-411b7cf5b9fc None admin] RBAC: Authorizing `identity:get_group(group_id=5ce458bca7f34105b7709b4e4447bb29)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:14.085027 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c77cee-654b-4a69-8074-411b7cf5b9fc None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:14.088914 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 194/381] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Jul 17 16:40:13 2022] GET /identity/v3/groups/5ce458bca7f34105b7709b4e4447bb29 => generated 229 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:14.094182 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c77cee-654b-4a69-8074-411b7cf5b9fc None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:14.145588 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:14.176920 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:14.263775 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b02f902-7c94-4248-acdf-4e9f907d903c None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:14.335816 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb99f68-6f61-4d57-9344-9d6a08a85717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:14.336676 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb99f68-6f61-4d57-9344-9d6a08a85717 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:14.337006 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb99f68-6f61-4d57-9344-9d6a08a85717 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:14.337293 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb99f68-6f61-4d57-9344-9d6a08a85717 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:14.343278 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c77cee-654b-4a69-8074-411b7cf5b9fc None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:14.350423 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb99f68-6f61-4d57-9344-9d6a08a85717 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:14.359042 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-1bb99f68-6f61-4d57-9344-9d6a08a85717 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 17 16:40:14.359042 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:14.359042 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:14.359042 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:14.359042 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:14.359042 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:14.359042 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:14.359042 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:14.359042 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:14.359042 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:14.359042 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:14.359042 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:14.359042 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:14.359042 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:14.359042 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:14.359042 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:14.359997 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:14.359997 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:14.359997 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:14.359997 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:14.359997 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:14.359997 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:14.359997 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:14.359997 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:14.359997 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:14.359997 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:14.359997 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:14.359997 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:14.359997 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:14.359997 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:14.359997 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:14.359997 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:14.365579 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:14.365579 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:14.365579 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:14.365579 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:14.365579 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:14.365579 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:14.365579 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:14.365579 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:14.365579 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:14.365579 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:14.365579 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 17 16:40:14.365579 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:14.365579 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 188/382] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 17 16:40:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:40:14.373366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb99f68-6f61-4d57-9344-9d6a08a85717 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:14.415671 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d9221-63a5-4ecb-98ab-07144bf4799f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:14.416923 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d9221-63a5-4ecb-98ab-07144bf4799f None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:14.418374 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d9221-63a5-4ecb-98ab-07144bf4799f None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:14.421930 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d9221-63a5-4ecb-98ab-07144bf4799f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:14.423074 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d9221-63a5-4ecb-98ab-07144bf4799f None admin] RBAC: Adding query filter params (group.id=5ce458bca7f34105b7709b4e4447bb29, role.id=1025051975f64f689effa58dc4259e36, scope.project.id=3b2ec585f5d6476389a82d18a223a808) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:14.423600 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d9221-63a5-4ecb-98ab-07144bf4799f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:14.426200 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d9221-63a5-4ecb-98ab-07144bf4799f None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:14.439394 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 195/383] 10.222.0.40 () {56 vars in 1376 bytes} [Sun Jul 17 16:40:14 2022] GET /identity/v3/role_assignments?group.id=5ce458bca7f34105b7709b4e4447bb29&scope.project.id=3b2ec585f5d6476389a82d18a223a808&role.id=1025051975f64f689effa58dc4259e36 => generated 260 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:14.566067 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb99f68-6f61-4d57-9344-9d6a08a85717 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:14.600076 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:14.626212 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f366c01-6287-49d6-b375-63cf8b850601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:14.627473 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f366c01-6287-49d6-b375-63cf8b850601 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:14.628002 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f366c01-6287-49d6-b375-63cf8b850601 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:14.628502 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f366c01-6287-49d6-b375-63cf8b850601 None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:14.629589 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f366c01-6287-49d6-b375-63cf8b850601 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:14.630152 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f366c01-6287-49d6-b375-63cf8b850601 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:14.632193 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f366c01-6287-49d6-b375-63cf8b850601 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:14.634314 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec349e-2fd3-4b0c-ba4a-416222457820 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:14.634825 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec349e-2fd3-4b0c-ba4a-416222457820 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:14.635326 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec349e-2fd3-4b0c-ba4a-416222457820 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:14.648818 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 196/385] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:14 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:40:14.649313 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 189/385] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:40:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:14.653969 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f366c01-6287-49d6-b375-63cf8b850601 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:14.656982 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bf32f-3454-4ac0-a9fa-51fb600f67cf None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:14.656982 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bf32f-3454-4ac0-a9fa-51fb600f67cf None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:14.656982 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bf32f-3454-4ac0-a9fa-51fb600f67cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:14.722976 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-124bf32f-3454-4ac0-a9fa-51fb600f67cf None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:14.725173 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-124bf32f-3454-4ac0-a9fa-51fb600f67cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:14.778920 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f366c01-6287-49d6-b375-63cf8b850601 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:14.820758 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7603718-a5a3-43ad-8df8-541ff56d4257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:14.822021 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7603718-a5a3-43ad-8df8-541ff56d4257 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:14.822534 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7603718-a5a3-43ad-8df8-541ff56d4257 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:14.823216 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7603718-a5a3-43ad-8df8-541ff56d4257 None admin] PATH_INFO: `/v3/groups/490d3612ce324804bb32d642c6b92f03` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:14.824932 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7603718-a5a3-43ad-8df8-541ff56d4257 None admin] RBAC: Authorizing `identity:get_group(group_id=490d3612ce324804bb32d642c6b92f03)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:14.826654 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7603718-a5a3-43ad-8df8-541ff56d4257 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:14.828890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 190/386] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Jul 17 16:40:14 2022] GET /identity/v3/groups/490d3612ce324804bb32d642c6b92f03 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:14.834382 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7603718-a5a3-43ad-8df8-541ff56d4257 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:14.943266 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:14.971474 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 197/387] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:15.022865 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124bf32f-3454-4ac0-a9fa-51fb600f67cf None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:15.050225 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:15.058873 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7603718-a5a3-43ad-8df8-541ff56d4257 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:15.146776 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607518be-12a2-4575-a2c5-f114e686c84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:15.148208 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607518be-12a2-4575-a2c5-f114e686c84f None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:15.148714 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607518be-12a2-4575-a2c5-f114e686c84f None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:15.149249 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607518be-12a2-4575-a2c5-f114e686c84f None admin] PATH_INFO: `/v3/projects/fde6b97a4d09416a81257067f1d1b919` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:15.152955 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607518be-12a2-4575-a2c5-f114e686c84f None admin] RBAC: Authorizing `identity:get_project(project_id=fde6b97a4d09416a81257067f1d1b919)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:15.155641 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607518be-12a2-4575-a2c5-f114e686c84f None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:15.159642 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 191/388] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:40:14 2022] GET /identity/v3/projects/fde6b97a4d09416a81257067f1d1b919 => generated 298 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:15.171024 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607518be-12a2-4575-a2c5-f114e686c84f None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:15.332662 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124bf32f-3454-4ac0-a9fa-51fb600f67cf None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:15.382015 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607518be-12a2-4575-a2c5-f114e686c84f None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:15.413656 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a882b01-c078-4c28-b3c1-95830055a9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:15.415210 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a882b01-c078-4c28-b3c1-95830055a9c9 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:15.415711 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a882b01-c078-4c28-b3c1-95830055a9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:15.416281 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a882b01-c078-4c28-b3c1-95830055a9c9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:15.426321 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:15.445991 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a882b01-c078-4c28-b3c1-95830055a9c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:15.449428 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad124c54-655b-4046-b261-8be058de52c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:15.450952 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad124c54-655b-4046-b261-8be058de52c6 None admin] REQUEST_METHOD: `PUT` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:15.451452 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad124c54-655b-4046-b261-8be058de52c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:15.451949 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad124c54-655b-4046-b261-8be058de52c6 None admin] PATH_INFO: `/v3/projects/fde6b97a4d09416a81257067f1d1b919/groups/490d3612ce324804bb32d642c6b92f03/roles/71cf67eb04704663aff719f6e3804f1a` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:15.459740 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-6a882b01-c078-4c28-b3c1-95830055a9c9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:40:15.459740 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:15.459740 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:15.459740 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:15.459740 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:15.459740 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:15.459740 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:15.459740 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:15.459740 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:15.459740 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:15.459740 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:15.459740 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:15.459740 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:15.459740 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:15.459740 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:15.459740 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:15.460674 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:15.460674 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:15.460674 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:15.460674 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:15.460674 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:15.460674 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:15.460674 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:15.460674 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:15.460674 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:15.460674 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:15.460674 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:15.460674 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:15.460674 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:15.460674 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:15.460674 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:15.460674 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:15.462036 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:15.462036 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:15.462036 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:15.462036 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:15.462036 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:15.462036 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:15.462036 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:15.462036 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:15.462036 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:15.462036 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:15.462036 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:40:15.462036 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:15.462036 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 198/389] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:40:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 439 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:15.466833 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad124c54-655b-4046-b261-8be058de52c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=fde6b97a4d09416a81257067f1d1b919, group_id=490d3612ce324804bb32d642c6b92f03, role_id=71cf67eb04704663aff719f6e3804f1a)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:15.468322 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad124c54-655b-4046-b261-8be058de52c6 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:15.475524 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a882b01-c078-4c28-b3c1-95830055a9c9 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:15.542772 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 192/390] 10.222.0.40 () {58 vars in 1442 bytes} [Sun Jul 17 16:40:15 2022] PUT /identity/v3/projects/fde6b97a4d09416a81257067f1d1b919/groups/490d3612ce324804bb32d642c6b92f03/roles/71cf67eb04704663aff719f6e3804f1a => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 17 16:40:15.565697 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:15.601305 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a882b01-c078-4c28-b3c1-95830055a9c9 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:15.658046 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e62d28-0211-41eb-ad4b-bd384d0890d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:15.659617 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e62d28-0211-41eb-ad4b-bd384d0890d3 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:15.660049 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e62d28-0211-41eb-ad4b-bd384d0890d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:15.660470 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e62d28-0211-41eb-ad4b-bd384d0890d3 None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:15.661578 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e62d28-0211-41eb-ad4b-bd384d0890d3 None admin] RBAC: Adding query filter params (name=member) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:15.662033 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e62d28-0211-41eb-ad4b-bd384d0890d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:15.663608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e62d28-0211-41eb-ad4b-bd384d0890d3 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:15.677908 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 199/391] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 17 16:40:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:40:15.687495 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad124c54-655b-4046-b261-8be058de52c6 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:15.836637 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad124c54-655b-4046-b261-8be058de52c6 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:15.876622 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d3bbae-904d-45bc-a0de-0428f4982d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:15.881794 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3bbae-904d-45bc-a0de-0428f4982d25 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:15.881794 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3bbae-904d-45bc-a0de-0428f4982d25 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:15.881794 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3bbae-904d-45bc-a0de-0428f4982d25 None admin] PATH_INFO: `/v3/groups/490d3612ce324804bb32d642c6b92f03` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:15.881794 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d3bbae-904d-45bc-a0de-0428f4982d25 None admin] RBAC: Authorizing `identity:get_group(group_id=490d3612ce324804bb32d642c6b92f03)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:15.881794 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d3bbae-904d-45bc-a0de-0428f4982d25 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:15.883341 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 193/392] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Jul 17 16:40:15 2022] GET /identity/v3/groups/490d3612ce324804bb32d642c6b92f03 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:40:15.892347 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e62d28-0211-41eb-ad4b-bd384d0890d3 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:16.025186 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f2719-8437-410e-be65-89367b7bffac None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:16.025508 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f2719-8437-410e-be65-89367b7bffac None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:16.025747 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f2719-8437-410e-be65-89367b7bffac None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:16.027399 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 194/393] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:40:16.033733 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d0b7a-1687-42fc-9ed4-9f2172adfcf9 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:16.034262 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d0b7a-1687-42fc-9ed4-9f2172adfcf9 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:16.034783 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d0b7a-1687-42fc-9ed4-9f2172adfcf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:16.091760 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-fc0d0b7a-1687-42fc-9ed4-9f2172adfcf9 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:16.093901 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-fc0d0b7a-1687-42fc-9ed4-9f2172adfcf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:16.116513 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e62d28-0211-41eb-ad4b-bd384d0890d3 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:16.169220 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:16.173102 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:16.182498 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2913e0fe-978e-4800-8efe-3a58273fbf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:16.188937 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913e0fe-978e-4800-8efe-3a58273fbf6d None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:16.188937 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913e0fe-978e-4800-8efe-3a58273fbf6d None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:16.188937 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913e0fe-978e-4800-8efe-3a58273fbf6d None admin] PATH_INFO: `/v3/projects/fde6b97a4d09416a81257067f1d1b919` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:16.188937 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2913e0fe-978e-4800-8efe-3a58273fbf6d None admin] RBAC: Authorizing `identity:get_project(project_id=fde6b97a4d09416a81257067f1d1b919)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:16.188937 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2913e0fe-978e-4800-8efe-3a58273fbf6d None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:16.188937 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 200/394] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:40:15 2022] GET /identity/v3/projects/fde6b97a4d09416a81257067f1d1b919 => generated 298 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:16.196559 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2913e0fe-978e-4800-8efe-3a58273fbf6d None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:16.200262 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 195/395] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 17 16:40:16.225399 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d0b7a-1687-42fc-9ed4-9f2172adfcf9 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:16.336402 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d0b7a-1687-42fc-9ed4-9f2172adfcf9 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:16.378133 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c3e2e-19c9-42bc-bc99-ba01c861fdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:16.379255 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c3e2e-19c9-42bc-bc99-ba01c861fdf7 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:16.379643 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c3e2e-19c9-42bc-bc99-ba01c861fdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:16.380082 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c3e2e-19c9-42bc-bc99-ba01c861fdf7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:16.392598 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426c3e2e-19c9-42bc-bc99-ba01c861fdf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:16.401403 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-426c3e2e-19c9-42bc-bc99-ba01c861fdf7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:40:16.401403 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:16.401403 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:16.401403 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:16.401403 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:16.401403 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:16.401403 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:16.401403 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:16.401403 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:16.401403 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:16.401403 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:16.401403 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:16.401403 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:16.401403 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:16.401403 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:16.401403 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:16.402288 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:16.402288 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:16.402288 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:16.402288 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:16.402288 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:16.402288 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:16.402288 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:16.402288 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:16.402288 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:16.402288 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:16.402288 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:16.402288 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:16.402288 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:16.402288 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:16.402288 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:16.402288 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:16.403571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:16.403571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:16.403571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:16.403571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:16.403571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:16.403571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:16.403571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:16.403571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:16.403571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:16.403571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:16.403571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:40:16.403571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:16.404460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2913e0fe-978e-4800-8efe-3a58273fbf6d None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:16.404618 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 196/396] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:40:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:16.411699 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c3e2e-19c9-42bc-bc99-ba01c861fdf7 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:16.455180 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:16.518643 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ff1f4-5e17-48ff-b7d1-b2e348d78511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:16.519561 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ff1f4-5e17-48ff-b7d1-b2e348d78511 None admin] REQUEST_METHOD: `PUT` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:16.519846 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ff1f4-5e17-48ff-b7d1-b2e348d78511 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:16.520121 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ff1f4-5e17-48ff-b7d1-b2e348d78511 None admin] PATH_INFO: `/v3/projects/fde6b97a4d09416a81257067f1d1b919/groups/490d3612ce324804bb32d642c6b92f03/roles/a47f5df8505f49419d0acb4f0f287610` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:16.526375 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ff1f4-5e17-48ff-b7d1-b2e348d78511 None admin] RBAC: Authorizing `identity:create_grant(project_id=fde6b97a4d09416a81257067f1d1b919, group_id=490d3612ce324804bb32d642c6b92f03, role_id=a47f5df8505f49419d0acb4f0f287610)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:16.527862 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ff1f4-5e17-48ff-b7d1-b2e348d78511 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:16.532144 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c3e2e-19c9-42bc-bc99-ba01c861fdf7 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:16.572007 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2afb044-52b3-4516-a21b-399b9294681a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:16.572684 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afb044-52b3-4516-a21b-399b9294681a None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:16.573061 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afb044-52b3-4516-a21b-399b9294681a None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:16.573346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afb044-52b3-4516-a21b-399b9294681a None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:16.574113 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2afb044-52b3-4516-a21b-399b9294681a None admin] RBAC: Adding query filter params (name=member) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:16.574431 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2afb044-52b3-4516-a21b-399b9294681a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:16.575880 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2afb044-52b3-4516-a21b-399b9294681a None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:16.585589 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 197/397] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 17 16:40:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:16.587011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9057f8-50dc-4e70-b8b0-a77c7fe95a39 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:16.587245 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9057f8-50dc-4e70-b8b0-a77c7fe95a39 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:16.587528 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9057f8-50dc-4e70-b8b0-a77c7fe95a39 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:16.589040 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 198/398] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:16.597324 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b4840-567d-494f-be90-61f16e075846 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:16.597658 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b4840-567d-494f-be90-61f16e075846 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:16.597903 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b4840-567d-494f-be90-61f16e075846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:16.606513 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 201/399] 10.222.0.40 () {58 vars in 1442 bytes} [Sun Jul 17 16:40:16 2022] PUT /identity/v3/projects/fde6b97a4d09416a81257067f1d1b919/groups/490d3612ce324804bb32d642c6b92f03/roles/a47f5df8505f49419d0acb4f0f287610 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 17 16:40:16.608306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ff1f4-5e17-48ff-b7d1-b2e348d78511 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:16.617527 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-d34b4840-567d-494f-be90-61f16e075846 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:16.618804 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-d34b4840-567d-494f-be90-61f16e075846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:16.683595 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:16.754592 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 199/400] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:16.757266 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ff1f4-5e17-48ff-b7d1-b2e348d78511 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:16.795009 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b4840-567d-494f-be90-61f16e075846 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:16.807178 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f9e15-7716-4ad7-85b3-ed498d6a8e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:16.809274 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f9e15-7716-4ad7-85b3-ed498d6a8e99 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:16.810097 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f9e15-7716-4ad7-85b3-ed498d6a8e99 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:16.810850 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f9e15-7716-4ad7-85b3-ed498d6a8e99 None admin] PATH_INFO: `/v3/groups/490d3612ce324804bb32d642c6b92f03` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:16.813359 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f9e15-7716-4ad7-85b3-ed498d6a8e99 None admin] RBAC: Authorizing `identity:get_group(group_id=490d3612ce324804bb32d642c6b92f03)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:16.817567 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f9e15-7716-4ad7-85b3-ed498d6a8e99 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:16.821539 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 202/401] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Jul 17 16:40:16 2022] GET /identity/v3/groups/490d3612ce324804bb32d642c6b92f03 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:16.828326 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f9e15-7716-4ad7-85b3-ed498d6a8e99 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:16.835088 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:16.965348 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b4840-567d-494f-be90-61f16e075846 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:16.978991 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f9e15-7716-4ad7-85b3-ed498d6a8e99 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:17.013183 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:17.019020 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb89623-3180-4cab-8a33-bbb629096be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:17.020231 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb89623-3180-4cab-8a33-bbb629096be6 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:17.020713 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb89623-3180-4cab-8a33-bbb629096be6 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:17.021218 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb89623-3180-4cab-8a33-bbb629096be6 None admin] PATH_INFO: `/v3/projects/6704fc6d36e84a078282ac2057784505` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:17.022701 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb89623-3180-4cab-8a33-bbb629096be6 None admin] RBAC: Authorizing `identity:get_project(project_id=6704fc6d36e84a078282ac2057784505)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:17.024450 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb89623-3180-4cab-8a33-bbb629096be6 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:17.026862 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 203/402] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:40:16 2022] GET /identity/v3/projects/6704fc6d36e84a078282ac2057784505 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:17.037895 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb89623-3180-4cab-8a33-bbb629096be6 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:17.061045 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df101332-b5fe-4a2f-b12e-54a9995ebdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:17.061880 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df101332-b5fe-4a2f-b12e-54a9995ebdb9 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:17.062168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df101332-b5fe-4a2f-b12e-54a9995ebdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:17.062448 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df101332-b5fe-4a2f-b12e-54a9995ebdb9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:17.096117 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df101332-b5fe-4a2f-b12e-54a9995ebdb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:17.104624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-df101332-b5fe-4a2f-b12e-54a9995ebdb9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 17 16:40:17.104624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:17.104624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:17.104624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:17.104624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:17.104624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:17.104624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:17.104624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:17.104624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:17.104624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:17.104624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:17.104624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:17.104624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:17.104624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:17.104624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:17.104624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:17.105802 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:17.105802 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:17.105802 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:17.105802 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:17.105802 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:17.105802 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:17.105802 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:17.105802 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:17.105802 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:17.105802 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:17.105802 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:17.105802 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:17.105802 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:17.105802 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:17.105802 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:17.105802 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:17.106586 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:17.106586 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:17.106586 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:17.106586 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:17.106586 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:17.106586 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:17.106586 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:17.106586 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:17.106586 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:17.106586 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:17.106586 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 17 16:40:17.106586 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:17.110064 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 200/403] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 17 16:40:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:17.116714 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df101332-b5fe-4a2f-b12e-54a9995ebdb9 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:17.196443 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb89623-3180-4cab-8a33-bbb629096be6 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:17.274462 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a8bc65-33e4-4ade-a1a9-6eb9ea2561da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:17.275361 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8bc65-33e4-4ade-a1a9-6eb9ea2561da None admin] REQUEST_METHOD: `PUT` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:17.275648 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8bc65-33e4-4ade-a1a9-6eb9ea2561da None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:17.275925 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8bc65-33e4-4ade-a1a9-6eb9ea2561da None admin] PATH_INFO: `/v3/projects/6704fc6d36e84a078282ac2057784505/groups/490d3612ce324804bb32d642c6b92f03/roles/a47f5df8505f49419d0acb4f0f287610` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:17.279797 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a8bc65-33e4-4ade-a1a9-6eb9ea2561da None admin] RBAC: Authorizing `identity:create_grant(project_id=6704fc6d36e84a078282ac2057784505, group_id=490d3612ce324804bb32d642c6b92f03, role_id=a47f5df8505f49419d0acb4f0f287610)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:17.281304 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a8bc65-33e4-4ade-a1a9-6eb9ea2561da None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:17.300590 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:17.312900 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df101332-b5fe-4a2f-b12e-54a9995ebdb9 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:17.362921 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 204/404] 10.222.0.40 () {58 vars in 1442 bytes} [Sun Jul 17 16:40:17 2022] PUT /identity/v3/projects/6704fc6d36e84a078282ac2057784505/groups/490d3612ce324804bb32d642c6b92f03/roles/a47f5df8505f49419d0acb4f0f287610 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 17 16:40:17.362921 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047bd4b-1363-462c-85d9-a11f68d92b77 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:17.362921 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047bd4b-1363-462c-85d9-a11f68d92b77 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:17.362921 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047bd4b-1363-462c-85d9-a11f68d92b77 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:17.363992 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 205/405] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:17.366592 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01026640-79fc-42b7-9ed4-f29a0730cb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:17.366592 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01026640-79fc-42b7-9ed4-f29a0730cb7a None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:17.366592 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01026640-79fc-42b7-9ed4-f29a0730cb7a None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:17.367153 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01026640-79fc-42b7-9ed4-f29a0730cb7a None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:17.368483 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01026640-79fc-42b7-9ed4-f29a0730cb7a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:17.368483 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01026640-79fc-42b7-9ed4-f29a0730cb7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:17.370009 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01026640-79fc-42b7-9ed4-f29a0730cb7a None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:17.382557 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 201/406] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:40:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:17.392021 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6047bd4b-1363-462c-85d9-a11f68d92b77 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:17.406597 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c383a5-b728-470a-8fb1-6583c35f0665 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:17.407190 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c383a5-b728-470a-8fb1-6583c35f0665 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:17.407190 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c383a5-b728-470a-8fb1-6583c35f0665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:17.438300 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-67c383a5-b728-470a-8fb1-6583c35f0665 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:17.440858 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-67c383a5-b728-470a-8fb1-6583c35f0665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:17.475027 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:17.546588 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6047bd4b-1363-462c-85d9-a11f68d92b77 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:17.599085 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a644dbe7-5ac8-4219-b588-651e5ff386b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:17.600763 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644dbe7-5ac8-4219-b588-651e5ff386b7 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:17.601351 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644dbe7-5ac8-4219-b588-651e5ff386b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:17.601844 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644dbe7-5ac8-4219-b588-651e5ff386b7 None admin] PATH_INFO: `/v3/groups/490d3612ce324804bb32d642c6b92f03` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:17.603634 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a644dbe7-5ac8-4219-b588-651e5ff386b7 None admin] RBAC: Authorizing `identity:get_group(group_id=490d3612ce324804bb32d642c6b92f03)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:17.606259 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a644dbe7-5ac8-4219-b588-651e5ff386b7 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:17.613441 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 206/407] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Jul 17 16:40:17 2022] GET /identity/v3/groups/490d3612ce324804bb32d642c6b92f03 => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:40:17.616275 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a644dbe7-5ac8-4219-b588-651e5ff386b7 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:17.618333 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 202/408] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 17 16:40:17.669247 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c383a5-b728-470a-8fb1-6583c35f0665 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:17.775300 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a644dbe7-5ac8-4219-b588-651e5ff386b7 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:17.815342 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:17.824819 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:17.851470 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f3e0ba-e29b-4df8-8330-261d2328d7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:17.852361 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3e0ba-e29b-4df8-8330-261d2328d7b3 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:17.852647 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3e0ba-e29b-4df8-8330-261d2328d7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:17.852957 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f3e0ba-e29b-4df8-8330-261d2328d7b3 None admin] PATH_INFO: `/v3/projects/6704fc6d36e84a078282ac2057784505` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:17.854393 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f3e0ba-e29b-4df8-8330-261d2328d7b3 None admin] RBAC: Authorizing `identity:get_project(project_id=6704fc6d36e84a078282ac2057784505)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:17.856082 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f3e0ba-e29b-4df8-8330-261d2328d7b3 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:17.857890 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 207/409] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:40:17 2022] GET /identity/v3/projects/6704fc6d36e84a078282ac2057784505 => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:17.859868 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c383a5-b728-470a-8fb1-6583c35f0665 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:17.870696 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f3e0ba-e29b-4df8-8330-261d2328d7b3 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:17.944193 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b15ae3-f49c-4a67-baca-b1f544f177fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:17.945046 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b15ae3-f49c-4a67-baca-b1f544f177fa None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:17.945584 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b15ae3-f49c-4a67-baca-b1f544f177fa None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:17.945872 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b15ae3-f49c-4a67-baca-b1f544f177fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:17.966547 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b15ae3-f49c-4a67-baca-b1f544f177fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:17.980981 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-26b15ae3-f49c-4a67-baca-b1f544f177fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:40:17.980981 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:17.980981 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:17.980981 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:17.980981 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:17.980981 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:17.980981 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:17.980981 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:17.980981 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:17.980981 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:17.980981 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:17.980981 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:17.980981 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:17.980981 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:17.980981 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:17.980981 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:17.982672 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:17.982672 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:17.982672 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:17.982672 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:17.982672 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:17.982672 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:17.982672 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:17.982672 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:17.982672 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:17.982672 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:17.982672 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:17.982672 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:17.982672 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:17.982672 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:17.982672 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:17.982672 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:17.983995 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:17.983995 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:17.983995 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:17.983995 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:17.983995 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:17.983995 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:17.983995 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:17.983995 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:17.983995 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:17.983995 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:17.983995 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:40:17.983995 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:17.983995 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 203/410] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:40:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:40:17.992710 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b15ae3-f49c-4a67-baca-b1f544f177fa None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:18.069697 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f3e0ba-e29b-4df8-8330-261d2328d7b3 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:18.158953 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f7af3-5317-48de-ab69-b8963019953a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:18.160315 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f7af3-5317-48de-ab69-b8963019953a None admin] REQUEST_METHOD: `PUT` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:18.160859 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f7af3-5317-48de-ab69-b8963019953a None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:18.161390 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f7af3-5317-48de-ab69-b8963019953a None admin] PATH_INFO: `/v3/projects/6704fc6d36e84a078282ac2057784505/groups/490d3612ce324804bb32d642c6b92f03/roles/71cf67eb04704663aff719f6e3804f1a` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:18.164074 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f7af3-5317-48de-ab69-b8963019953a None admin] RBAC: Authorizing `identity:create_grant(project_id=6704fc6d36e84a078282ac2057784505, group_id=490d3612ce324804bb32d642c6b92f03, role_id=71cf67eb04704663aff719f6e3804f1a)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:18.165872 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f7af3-5317-48de-ab69-b8963019953a None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:18.192146 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b15ae3-f49c-4a67-baca-b1f544f177fa None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:18.233879 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb1e0f-ddc8-4956-bbdc-11f9a0dab0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:18.234578 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb1e0f-ddc8-4956-bbdc-11f9a0dab0bc None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:18.234868 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb1e0f-ddc8-4956-bbdc-11f9a0dab0bc None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:18.235152 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb1e0f-ddc8-4956-bbdc-11f9a0dab0bc None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:18.236235 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cb1e0f-ddc8-4956-bbdc-11f9a0dab0bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:18.237089 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cb1e0f-ddc8-4956-bbdc-11f9a0dab0bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:18.238640 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cb1e0f-ddc8-4956-bbdc-11f9a0dab0bc None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:18.249714 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 208/411] 10.222.0.40 () {58 vars in 1442 bytes} [Sun Jul 17 16:40:17 2022] PUT /identity/v3/projects/6704fc6d36e84a078282ac2057784505/groups/490d3612ce324804bb32d642c6b92f03/roles/71cf67eb04704663aff719f6e3804f1a => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 17 16:40:18.255347 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 204/412] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:40:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:18.255810 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8da7bb-c9c1-48bd-af8a-8a7886f27cfa None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:18.256412 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8da7bb-c9c1-48bd-af8a-8a7886f27cfa None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:18.256815 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8da7bb-c9c1-48bd-af8a-8a7886f27cfa None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:18.259389 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 209/413] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:18.260218 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb1e0f-ddc8-4956-bbdc-11f9a0dab0bc None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:18.264639 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:18.275017 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0fc8c-8cf3-4681-8da2-7941ff59e5cb None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:18.276195 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0fc8c-8cf3-4681-8da2-7941ff59e5cb None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:18.276625 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0fc8c-8cf3-4681-8da2-7941ff59e5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:18.324486 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-5bf0fc8c-8cf3-4681-8da2-7941ff59e5cb None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:18.327820 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-5bf0fc8c-8cf3-4681-8da2-7941ff59e5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:18.426715 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:18.470593 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 210/414] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 17 16:40:18.490782 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb1e0f-ddc8-4956-bbdc-11f9a0dab0bc None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:18.503935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0fc8c-8cf3-4681-8da2-7941ff59e5cb None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:18.536169 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd789bf-e2b5-409e-913d-030936c27230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:18.537593 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd789bf-e2b5-409e-913d-030936c27230 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:18.538143 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd789bf-e2b5-409e-913d-030936c27230 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:18.538622 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd789bf-e2b5-409e-913d-030936c27230 None admin] PATH_INFO: `/v3/groups/5ce458bca7f34105b7709b4e4447bb29` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:18.540182 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd789bf-e2b5-409e-913d-030936c27230 None admin] RBAC: Authorizing `identity:get_group(group_id=5ce458bca7f34105b7709b4e4447bb29)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:18.541958 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd789bf-e2b5-409e-913d-030936c27230 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:18.545906 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 205/415] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Jul 17 16:40:18 2022] GET /identity/v3/groups/5ce458bca7f34105b7709b4e4447bb29 => generated 229 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:18.549998 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd789bf-e2b5-409e-913d-030936c27230 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:18.659509 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0fc8c-8cf3-4681-8da2-7941ff59e5cb None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:18.702404 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:18.742863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80956400-f863-42e7-94a6-3517624c69be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:18.744055 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80956400-f863-42e7-94a6-3517624c69be None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:18.744570 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80956400-f863-42e7-94a6-3517624c69be None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:18.745148 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80956400-f863-42e7-94a6-3517624c69be None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:18.757263 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd789bf-e2b5-409e-913d-030936c27230 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:18.765667 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80956400-f863-42e7-94a6-3517624c69be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:18.783789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-80956400-f863-42e7-94a6-3517624c69be None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 17 16:40:18.783789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:18.783789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:18.783789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:18.783789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:18.783789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:18.783789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:18.783789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:18.783789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:18.783789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:18.783789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:18.783789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:18.783789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:18.783789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:18.783789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:18.783789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:18.784742 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:18.784742 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:18.784742 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:18.784742 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:18.784742 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:18.784742 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:18.784742 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:18.784742 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:18.784742 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:18.784742 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:18.784742 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:18.784742 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:18.784742 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:18.784742 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:18.784742 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:18.784742 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:18.785594 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:18.785594 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:18.785594 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:18.785594 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:18.785594 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:18.785594 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:18.785594 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:18.785594 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:18.785594 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:18.785594 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:18.785594 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 17 16:40:18.785594 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:18.790848 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 211/416] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 17 16:40:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:18.795639 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80956400-f863-42e7-94a6-3517624c69be None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:18.843109 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7f26d-cb7a-4a39-8d97-abb7ac961008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:18.844617 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7f26d-cb7a-4a39-8d97-abb7ac961008 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:18.845888 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7f26d-cb7a-4a39-8d97-abb7ac961008 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:18.846580 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7f26d-cb7a-4a39-8d97-abb7ac961008 None admin] PATH_INFO: `/v3/projects/3b2ec585f5d6476389a82d18a223a808` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:18.850053 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7f26d-cb7a-4a39-8d97-abb7ac961008 None admin] RBAC: Authorizing `identity:get_project(project_id=3b2ec585f5d6476389a82d18a223a808)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:18.851745 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7f26d-cb7a-4a39-8d97-abb7ac961008 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:18.855381 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 206/417] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:40:18 2022] GET /identity/v3/projects/3b2ec585f5d6476389a82d18a223a808 => generated 344 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:18.859742 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7f26d-cb7a-4a39-8d97-abb7ac961008 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:18.886723 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:18.989126 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80956400-f863-42e7-94a6-3517624c69be None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:19.029528 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7f26d-cb7a-4a39-8d97-abb7ac961008 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:19.030490 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811cf25c-53d6-4c0d-a2c0-7ba81696a853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:19.031416 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811cf25c-53d6-4c0d-a2c0-7ba81696a853 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:19.032198 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811cf25c-53d6-4c0d-a2c0-7ba81696a853 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:19.032681 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811cf25c-53d6-4c0d-a2c0-7ba81696a853 None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:19.034219 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811cf25c-53d6-4c0d-a2c0-7ba81696a853 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:19.034757 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811cf25c-53d6-4c0d-a2c0-7ba81696a853 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:19.036405 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811cf25c-53d6-4c0d-a2c0-7ba81696a853 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:19.045966 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 212/418] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:40:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:40:19.047656 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e309a-da08-4443-88f7-28ead6b9b3c7 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:19.048151 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e309a-da08-4443-88f7-28ead6b9b3c7 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:19.049377 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e309a-da08-4443-88f7-28ead6b9b3c7 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:19.051423 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 213/419] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:19.052386 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e309a-da08-4443-88f7-28ead6b9b3c7 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:19.078044 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300c457-b899-4598-a0cf-d434551e3516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:19.080902 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300c457-b899-4598-a0cf-d434551e3516 None admin] REQUEST_METHOD: `PUT` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:19.081227 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300c457-b899-4598-a0cf-d434551e3516 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:19.081516 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300c457-b899-4598-a0cf-d434551e3516 None admin] PATH_INFO: `/v3/projects/3b2ec585f5d6476389a82d18a223a808/groups/5ce458bca7f34105b7709b4e4447bb29/roles/1025051975f64f689effa58dc4259e36` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:19.084202 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300c457-b899-4598-a0cf-d434551e3516 None admin] RBAC: Authorizing `identity:create_grant(project_id=3b2ec585f5d6476389a82d18a223a808, group_id=5ce458bca7f34105b7709b4e4447bb29, role_id=1025051975f64f689effa58dc4259e36)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:19.085958 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300c457-b899-4598-a0cf-d434551e3516 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:19.156037 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:19.165649 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 207/420] 10.222.0.40 () {58 vars in 1442 bytes} [Sun Jul 17 16:40:18 2022] PUT /identity/v3/projects/3b2ec585f5d6476389a82d18a223a808/groups/5ce458bca7f34105b7709b4e4447bb29/roles/1025051975f64f689effa58dc4259e36 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 17 16:40:19.167400 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599456d1-8fb7-487a-98bd-e6b7a813736d None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:19.167943 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599456d1-8fb7-487a-98bd-e6b7a813736d None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:19.168475 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599456d1-8fb7-487a-98bd-e6b7a813736d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:19.171615 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e309a-da08-4443-88f7-28ead6b9b3c7 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:19.176515 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:19.197030 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-599456d1-8fb7-487a-98bd-e6b7a813736d None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:19.199321 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-599456d1-8fb7-487a-98bd-e6b7a813736d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:19.253943 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e6e9a-6c32-4f7a-8b57-bca3c38120fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:19.254529 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e6e9a-6c32-4f7a-8b57-bca3c38120fc None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:19.254818 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e6e9a-6c32-4f7a-8b57-bca3c38120fc None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:19.255096 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e6e9a-6c32-4f7a-8b57-bca3c38120fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:19.292616 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 208/421] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:19.323686 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599456d1-8fb7-487a-98bd-e6b7a813736d None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:19.425490 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e6e9a-6c32-4f7a-8b57-bca3c38120fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:19.427022 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e6e9a-6c32-4f7a-8b57-bca3c38120fc None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:19.454045 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599456d1-8fb7-487a-98bd-e6b7a813736d None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:19.492424 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:19.509454 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3ab890-2bbb-4005-81d7-aea7fe6f6258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:19.511233 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3ab890-2bbb-4005-81d7-aea7fe6f6258 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:19.511990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3ab890-2bbb-4005-81d7-aea7fe6f6258 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:19.512489 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3ab890-2bbb-4005-81d7-aea7fe6f6258 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:19.530267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3ab890-2bbb-4005-81d7-aea7fe6f6258 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:19.541147 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-2a3ab890-2bbb-4005-81d7-aea7fe6f6258 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:40:19.541147 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:19.541147 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:19.541147 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:19.541147 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:19.541147 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:19.541147 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:19.541147 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:19.541147 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:19.541147 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:19.541147 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:19.541147 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:19.541147 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:19.541147 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:19.541147 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:19.541147 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:19.542106 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:19.542106 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:19.542106 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:19.542106 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:19.542106 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:19.542106 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:19.542106 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:19.542106 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:19.542106 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:19.542106 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:19.542106 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:19.542106 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:19.542106 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:19.542106 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:19.542106 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:19.542106 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:19.542893 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:19.542893 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:19.542893 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:19.542893 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:19.542893 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:19.542893 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:19.542893 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:19.542893 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:19.542893 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:19.542893 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:19.542893 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:40:19.542893 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:19.546598 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 209/422] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:40:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:19.553249 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3ab890-2bbb-4005-81d7-aea7fe6f6258 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:19.567834 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 214/423] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:40:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 17 16:40:19.575138 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e6e9a-6c32-4f7a-8b57-bca3c38120fc None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:19.579322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:19.670039 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3ab890-2bbb-4005-81d7-aea7fe6f6258 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:19.709043 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50998d5-a19d-4b0f-9c15-e57dfbcf43e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:19.710142 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50998d5-a19d-4b0f-9c15-e57dfbcf43e9 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:19.710630 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50998d5-a19d-4b0f-9c15-e57dfbcf43e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:19.711104 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50998d5-a19d-4b0f-9c15-e57dfbcf43e9 None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:19.712082 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50998d5-a19d-4b0f-9c15-e57dfbcf43e9 None admin] RBAC: Adding query filter params (name=member) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:19.712588 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50998d5-a19d-4b0f-9c15-e57dfbcf43e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:19.714213 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50998d5-a19d-4b0f-9c15-e57dfbcf43e9 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:19.724961 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 210/424] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 17 16:40:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:40:19.730947 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50998d5-a19d-4b0f-9c15-e57dfbcf43e9 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:19.787165 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:19.809094 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e6e9a-6c32-4f7a-8b57-bca3c38120fc None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:19.843561 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50998d5-a19d-4b0f-9c15-e57dfbcf43e9 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:19.855048 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15668117-2114-469b-af22-ce29c37b2985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:19.855935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15668117-2114-469b-af22-ce29c37b2985 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:19.856213 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15668117-2114-469b-af22-ce29c37b2985 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:19.856492 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15668117-2114-469b-af22-ce29c37b2985 None admin] PATH_INFO: `/v3/projects/fde6b97a4d09416a81257067f1d1b919` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:19.857765 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15668117-2114-469b-af22-ce29c37b2985 None admin] RBAC: Authorizing `identity:get_project(project_id=fde6b97a4d09416a81257067f1d1b919)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:19.859220 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15668117-2114-469b-af22-ce29c37b2985 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:19.862136 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 215/425] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:40:19 2022] GET /identity/v3/projects/fde6b97a4d09416a81257067f1d1b919 => generated 298 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:19.863124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2babf-de4a-418e-a216-8cd3c877fba0 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:19.863420 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2babf-de4a-418e-a216-8cd3c877fba0 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:19.863702 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e2babf-de4a-418e-a216-8cd3c877fba0 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:19.865163 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 216/426] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:19.869863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e2babf-de4a-418e-a216-8cd3c877fba0 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:19.886260 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a21d90-2a97-423e-8e0c-a7f59ea8b8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:19.887269 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a21d90-2a97-423e-8e0c-a7f59ea8b8f3 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:19.887752 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a21d90-2a97-423e-8e0c-a7f59ea8b8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:19.888250 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a21d90-2a97-423e-8e0c-a7f59ea8b8f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:19.947627 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:19.966280 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a21d90-2a97-423e-8e0c-a7f59ea8b8f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:19.968087 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a21d90-2a97-423e-8e0c-a7f59ea8b8f3 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:19.982740 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e2babf-de4a-418e-a216-8cd3c877fba0 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:20.025487 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3bec12-7394-453b-8e82-3f2b1c01fc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:20.026523 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3bec12-7394-453b-8e82-3f2b1c01fc6c None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:20.026909 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3bec12-7394-453b-8e82-3f2b1c01fc6c None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:20.027285 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3bec12-7394-453b-8e82-3f2b1c01fc6c None admin] PATH_INFO: `/v3/groups/490d3612ce324804bb32d642c6b92f03` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:20.028591 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3bec12-7394-453b-8e82-3f2b1c01fc6c None admin] RBAC: Authorizing `identity:get_group(group_id=490d3612ce324804bb32d642c6b92f03)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:20.030315 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3bec12-7394-453b-8e82-3f2b1c01fc6c None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:20.032311 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 217/427] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Jul 17 16:40:19 2022] GET /identity/v3/groups/490d3612ce324804bb32d642c6b92f03 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:20.034291 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6b410-4103-4de2-a50f-f1bd86efb17e None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:20.034955 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6b410-4103-4de2-a50f-f1bd86efb17e None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:20.035248 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6b410-4103-4de2-a50f-f1bd86efb17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:20.053905 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-52c6b410-4103-4de2-a50f-f1bd86efb17e None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:20.055180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-52c6b410-4103-4de2-a50f-f1bd86efb17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:20.061229 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:20.092904 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 211/428] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:40:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 17 16:40:20.094069 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a21d90-2a97-423e-8e0c-a7f59ea8b8f3 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:20.145192 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 218/429] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:20.146495 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c6b410-4103-4de2-a50f-f1bd86efb17e None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:20.228776 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a21d90-2a97-423e-8e0c-a7f59ea8b8f3 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:20.252883 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:20.280041 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18007a62-aff4-4b5a-8a8c-b35e008cff91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:20.280731 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18007a62-aff4-4b5a-8a8c-b35e008cff91 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:20.281054 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18007a62-aff4-4b5a-8a8c-b35e008cff91 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:20.281343 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18007a62-aff4-4b5a-8a8c-b35e008cff91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:20.282284 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18007a62-aff4-4b5a-8a8c-b35e008cff91 None admin] RBAC: Adding query filter params (group.id=490d3612ce324804bb32d642c6b92f03, role.id=71cf67eb04704663aff719f6e3804f1a, scope.project.id=fde6b97a4d09416a81257067f1d1b919) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:20.282606 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18007a62-aff4-4b5a-8a8c-b35e008cff91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:20.285063 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18007a62-aff4-4b5a-8a8c-b35e008cff91 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:20.300873 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 212/430] 10.222.0.40 () {56 vars in 1376 bytes} [Sun Jul 17 16:40:20 2022] GET /identity/v3/role_assignments?group.id=490d3612ce324804bb32d642c6b92f03&scope.project.id=fde6b97a4d09416a81257067f1d1b919&role.id=71cf67eb04704663aff719f6e3804f1a => generated 611 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:20.300873 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18007a62-aff4-4b5a-8a8c-b35e008cff91 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:20.302339 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c6b410-4103-4de2-a50f-f1bd86efb17e None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:20.353323 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:20.390360 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a0775-4612-489b-92e4-8c9a3b1a0e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:20.391308 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a0775-4612-489b-92e4-8c9a3b1a0e4c None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:20.391646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a0775-4612-489b-92e4-8c9a3b1a0e4c None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:20.391931 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a0775-4612-489b-92e4-8c9a3b1a0e4c None admin] PATH_INFO: `/v3/projects/fde6b97a4d09416a81257067f1d1b919` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:20.394757 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6a0775-4612-489b-92e4-8c9a3b1a0e4c None admin] RBAC: Authorizing `identity:get_project(project_id=fde6b97a4d09416a81257067f1d1b919)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:20.396305 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6a0775-4612-489b-92e4-8c9a3b1a0e4c None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:20.398221 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 219/431] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:40:20 2022] GET /identity/v3/projects/fde6b97a4d09416a81257067f1d1b919 => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:20.404048 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a0775-4612-489b-92e4-8c9a3b1a0e4c None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:20.526332 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18007a62-aff4-4b5a-8a8c-b35e008cff91 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:20.566288 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a0775-4612-489b-92e4-8c9a3b1a0e4c None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:20.578017 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a65f58-b7f8-48eb-91f9-6571ffffc90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:20.578940 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a65f58-b7f8-48eb-91f9-6571ffffc90e None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:20.579292 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a65f58-b7f8-48eb-91f9-6571ffffc90e None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:20.580587 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a65f58-b7f8-48eb-91f9-6571ffffc90e None admin] PATH_INFO: `/v3/roles/member` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:20.593244 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a65f58-b7f8-48eb-91f9-6571ffffc90e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:20.601629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-e1a65f58-b7f8-48eb-91f9-6571ffffc90e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:40:20.601629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:20.601629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:20.601629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:20.601629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:20.601629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:20.601629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:20.601629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:20.601629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:20.601629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:20.601629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:20.601629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:20.601629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:20.601629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:20.601629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:20.601629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:20.602560 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:20.602560 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:20.602560 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:20.602560 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:20.602560 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:20.602560 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:20.602560 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:20.602560 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:20.602560 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:20.602560 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:20.602560 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:20.602560 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:20.602560 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:20.602560 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:20.602560 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:20.602560 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:20.603416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:20.603416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:20.603416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:20.603416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:20.603416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:20.603416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:20.603416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:20.603416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:20.603416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:20.603416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:20.603416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 17 16:40:20.603416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:20.603416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 213/432] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:40:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:20.611827 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a65f58-b7f8-48eb-91f9-6571ffffc90e None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:20.617146 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8dee91-3939-44a5-ba41-f85199e003d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:20.618373 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8dee91-3939-44a5-ba41-f85199e003d4 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:20.619026 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8dee91-3939-44a5-ba41-f85199e003d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:20.619497 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8dee91-3939-44a5-ba41-f85199e003d4 None admin] PATH_INFO: `/v3/groups/490d3612ce324804bb32d642c6b92f03` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:20.621002 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8dee91-3939-44a5-ba41-f85199e003d4 None admin] RBAC: Authorizing `identity:get_group(group_id=490d3612ce324804bb32d642c6b92f03)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:20.623384 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8dee91-3939-44a5-ba41-f85199e003d4 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:20.626214 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 220/433] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Jul 17 16:40:20 2022] GET /identity/v3/groups/490d3612ce324804bb32d642c6b92f03 => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:20.633993 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8dee91-3939-44a5-ba41-f85199e003d4 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:20.648663 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:20.726259 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:20.756897 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a65f58-b7f8-48eb-91f9-6571ffffc90e None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:20.775172 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8dee91-3939-44a5-ba41-f85199e003d4 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:20.801364 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f97083-b05d-4eb4-87ad-8c0dbeb3a168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:20.802042 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f97083-b05d-4eb4-87ad-8c0dbeb3a168 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:20.802332 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f97083-b05d-4eb4-87ad-8c0dbeb3a168 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:20.802609 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f97083-b05d-4eb4-87ad-8c0dbeb3a168 None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:20.803368 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f97083-b05d-4eb4-87ad-8c0dbeb3a168 None admin] RBAC: Adding query filter params (name=member) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:20.803675 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f97083-b05d-4eb4-87ad-8c0dbeb3a168 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:20.805119 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f97083-b05d-4eb4-87ad-8c0dbeb3a168 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:20.819074 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 214/434] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 17 16:40:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:20.819761 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3213f33c-5932-426a-833d-53105d245388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:20.820635 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e43ee-2fea-40e6-bc31-70cb477d007b None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:20.820752 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213f33c-5932-426a-833d-53105d245388 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:20.820948 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e43ee-2fea-40e6-bc31-70cb477d007b None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:20.821040 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213f33c-5932-426a-833d-53105d245388 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:20.821228 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e43ee-2fea-40e6-bc31-70cb477d007b None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:20.821318 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213f33c-5932-426a-833d-53105d245388 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:20.822086 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3213f33c-5932-426a-833d-53105d245388 None admin] RBAC: Adding query filter params (group.id=490d3612ce324804bb32d642c6b92f03, role.id=a47f5df8505f49419d0acb4f0f287610, scope.project.id=fde6b97a4d09416a81257067f1d1b919) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:20.822406 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3213f33c-5932-426a-833d-53105d245388 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:20.822772 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 215/435] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:20.823861 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3213f33c-5932-426a-833d-53105d245388 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:20.825595 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e43ee-2fea-40e6-bc31-70cb477d007b None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:20.836856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 221/436] 10.222.0.40 () {56 vars in 1376 bytes} [Sun Jul 17 16:40:20 2022] GET /identity/v3/role_assignments?group.id=490d3612ce324804bb32d642c6b92f03&scope.project.id=fde6b97a4d09416a81257067f1d1b919&role.id=a47f5df8505f49419d0acb4f0f287610 => generated 611 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:20.836856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ccdba-47c7-4999-a1a3-20bfb38a83d6 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:20.836856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ccdba-47c7-4999-a1a3-20bfb38a83d6 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:20.836856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ccdba-47c7-4999-a1a3-20bfb38a83d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:20.860174 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-d56ccdba-47c7-4999-a1a3-20bfb38a83d6 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:20.861804 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-d56ccdba-47c7-4999-a1a3-20bfb38a83d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:20.955966 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:20.963128 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 222/437] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:20.987754 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ccdba-47c7-4999-a1a3-20bfb38a83d6 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:20.993312 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e43ee-2fea-40e6-bc31-70cb477d007b None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:21.042507 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16674c9-a4c0-48cf-85d8-8b70c910b1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:21.043177 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16674c9-a4c0-48cf-85d8-8b70c910b1dd None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:21.043465 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16674c9-a4c0-48cf-85d8-8b70c910b1dd None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:21.043745 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16674c9-a4c0-48cf-85d8-8b70c910b1dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:21.049407 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:21.138253 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ccdba-47c7-4999-a1a3-20bfb38a83d6 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:21.144411 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16674c9-a4c0-48cf-85d8-8b70c910b1dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:21.146332 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16674c9-a4c0-48cf-85d8-8b70c910b1dd None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:21.190042 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b147100d-b3b7-4252-bf58-edc2b186c0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:21.190785 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147100d-b3b7-4252-bf58-edc2b186c0bc None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:21.191075 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147100d-b3b7-4252-bf58-edc2b186c0bc None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:21.191356 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147100d-b3b7-4252-bf58-edc2b186c0bc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:21.204881 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b147100d-b3b7-4252-bf58-edc2b186c0bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:21.214553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-b147100d-b3b7-4252-bf58-edc2b186c0bc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 17 16:40:21.214553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:21.214553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:21.214553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:21.214553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:21.214553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:21.214553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:21.214553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:21.214553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:21.214553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:21.214553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:21.214553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:21.214553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:21.214553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:21.214553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:21.214553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:21.215549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:21.215549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:21.215549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:21.215549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:21.215549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:21.215549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:21.215549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:21.215549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:21.215549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:21.215549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:21.215549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:21.215549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:21.215549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:21.215549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:21.215549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:21.215549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:21.216342 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:21.216342 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:21.216342 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:21.216342 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:21.216342 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:21.216342 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:21.216342 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:21.216342 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:21.216342 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:21.216342 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:21.216342 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 17 16:40:21.216342 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:21.221346 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 223/438] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 17 16:40:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:40:21.222652 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b147100d-b3b7-4252-bf58-edc2b186c0bc None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:21.310952 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:21.317394 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 216/439] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:40:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 17 16:40:21.323653 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16674c9-a4c0-48cf-85d8-8b70c910b1dd None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:21.379122 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b147100d-b3b7-4252-bf58-edc2b186c0bc None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:21.388642 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:21.442931 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99638e86-4b34-43d1-a272-00cf266492c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:21.444831 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99638e86-4b34-43d1-a272-00cf266492c9 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:21.444831 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99638e86-4b34-43d1-a272-00cf266492c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:21.445350 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99638e86-4b34-43d1-a272-00cf266492c9 None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:21.446473 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99638e86-4b34-43d1-a272-00cf266492c9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:21.446473 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99638e86-4b34-43d1-a272-00cf266492c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:21.447806 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99638e86-4b34-43d1-a272-00cf266492c9 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:21.467466 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 224/440] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:40:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:21.480715 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99638e86-4b34-43d1-a272-00cf266492c9 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:21.500143 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16674c9-a4c0-48cf-85d8-8b70c910b1dd None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:21.557045 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db9f15-3c9e-44dd-b48f-7ca5703aef2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:21.557045 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db9f15-3c9e-44dd-b48f-7ca5703aef2b None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:21.557045 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db9f15-3c9e-44dd-b48f-7ca5703aef2b None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:21.558162 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db9f15-3c9e-44dd-b48f-7ca5703aef2b None admin] PATH_INFO: `/v3/projects/6704fc6d36e84a078282ac2057784505` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:21.559318 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57db9f15-3c9e-44dd-b48f-7ca5703aef2b None admin] RBAC: Authorizing `identity:get_project(project_id=6704fc6d36e84a078282ac2057784505)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:21.562200 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57db9f15-3c9e-44dd-b48f-7ca5703aef2b None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:21.562200 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 217/441] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:40:21 2022] GET /identity/v3/projects/6704fc6d36e84a078282ac2057784505 => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:40:21.572844 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db9f15-3c9e-44dd-b48f-7ca5703aef2b None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:21.665008 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99638e86-4b34-43d1-a272-00cf266492c9 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:21.705618 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:21.717897 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2057bf-f675-4ed3-b621-24ca19c26e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:21.718568 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2057bf-f675-4ed3-b621-24ca19c26e55 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:21.718857 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2057bf-f675-4ed3-b621-24ca19c26e55 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:21.719139 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2057bf-f675-4ed3-b621-24ca19c26e55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:21.744752 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db9f15-3c9e-44dd-b48f-7ca5703aef2b None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:21.749180 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:21.796150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2057bf-f675-4ed3-b621-24ca19c26e55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:21.797645 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2057bf-f675-4ed3-b621-24ca19c26e55 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:21.806071 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6209e7fd-dcd7-4721-b4e0-7f86f6055539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:21.807541 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209e7fd-dcd7-4721-b4e0-7f86f6055539 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:21.808039 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209e7fd-dcd7-4721-b4e0-7f86f6055539 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:21.808538 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209e7fd-dcd7-4721-b4e0-7f86f6055539 None admin] PATH_INFO: `/v3/groups/490d3612ce324804bb32d642c6b92f03` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:21.810669 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6209e7fd-dcd7-4721-b4e0-7f86f6055539 None admin] RBAC: Authorizing `identity:get_group(group_id=490d3612ce324804bb32d642c6b92f03)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:21.813346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6209e7fd-dcd7-4721-b4e0-7f86f6055539 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:21.816989 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 218/442] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Jul 17 16:40:21 2022] GET /identity/v3/groups/490d3612ce324804bb32d642c6b92f03 => generated 226 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:40:21.819179 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59444550-6532-4d4a-b763-b5f024f0ca93 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:21.819679 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59444550-6532-4d4a-b763-b5f024f0ca93 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:21.820430 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59444550-6532-4d4a-b763-b5f024f0ca93 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:21.823033 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 219/443] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:21.824404 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59444550-6532-4d4a-b763-b5f024f0ca93 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:21.915471 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 225/444] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:40:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 17 16:40:21.917611 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d29d49-340b-489c-b54c-b1e1358626bc None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:21.918103 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d29d49-340b-489c-b54c-b1e1358626bc None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:21.918592 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d29d49-340b-489c-b54c-b1e1358626bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:21.938073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-49d29d49-340b-489c-b54c-b1e1358626bc None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:21.940116 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-49d29d49-340b-489c-b54c-b1e1358626bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:21.958797 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:22.007264 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59444550-6532-4d4a-b763-b5f024f0ca93 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:22.014037 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 226/445] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:22.015055 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d29d49-340b-489c-b54c-b1e1358626bc None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:22.064381 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84a551-5459-47db-aa67-84a47a86b3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:22.065438 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84a551-5459-47db-aa67-84a47a86b3ec None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:22.065835 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84a551-5459-47db-aa67-84a47a86b3ec None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:22.066176 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84a551-5459-47db-aa67-84a47a86b3ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:22.067471 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84a551-5459-47db-aa67-84a47a86b3ec None admin] RBAC: Adding query filter params (group.id=490d3612ce324804bb32d642c6b92f03, role.id=a47f5df8505f49419d0acb4f0f287610, scope.project.id=6704fc6d36e84a078282ac2057784505) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:22.067471 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84a551-5459-47db-aa67-84a47a86b3ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:22.069121 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84a551-5459-47db-aa67-84a47a86b3ec None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:22.076532 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 220/446] 10.222.0.40 () {56 vars in 1376 bytes} [Sun Jul 17 16:40:21 2022] GET /identity/v3/role_assignments?group.id=490d3612ce324804bb32d642c6b92f03&scope.project.id=6704fc6d36e84a078282ac2057784505&role.id=a47f5df8505f49419d0acb4f0f287610 => generated 611 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:22.077675 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84a551-5459-47db-aa67-84a47a86b3ec None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:22.124536 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:22.161823 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d29d49-340b-489c-b54c-b1e1358626bc None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:22.218729 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc63f18-7b17-419b-94a1-b55b77edc6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:22.221116 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc63f18-7b17-419b-94a1-b55b77edc6c5 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:22.221651 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc63f18-7b17-419b-94a1-b55b77edc6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:22.222140 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc63f18-7b17-419b-94a1-b55b77edc6c5 None admin] PATH_INFO: `/v3/projects/6704fc6d36e84a078282ac2057784505` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:22.224399 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc63f18-7b17-419b-94a1-b55b77edc6c5 None admin] RBAC: Authorizing `identity:get_project(project_id=6704fc6d36e84a078282ac2057784505)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:22.227247 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc63f18-7b17-419b-94a1-b55b77edc6c5 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:22.230507 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 227/447] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:40:22 2022] GET /identity/v3/projects/6704fc6d36e84a078282ac2057784505 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:22.239131 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc63f18-7b17-419b-94a1-b55b77edc6c5 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:22.242477 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84a551-5459-47db-aa67-84a47a86b3ec None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:22.280369 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:22.290279 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433d9a5-6230-4a93-84dd-6ee9f88b2fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:22.299082 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433d9a5-6230-4a93-84dd-6ee9f88b2fab None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:22.299082 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433d9a5-6230-4a93-84dd-6ee9f88b2fab None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:22.299082 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433d9a5-6230-4a93-84dd-6ee9f88b2fab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:22.314854 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b433d9a5-6230-4a93-84dd-6ee9f88b2fab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:22.326747 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-b433d9a5-6230-4a93-84dd-6ee9f88b2fab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:40:22.326747 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:22.326747 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:22.326747 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:22.326747 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:22.326747 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:22.326747 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:22.326747 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:22.326747 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:22.326747 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:22.326747 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:22.326747 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:22.326747 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:22.326747 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:22.326747 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:22.326747 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:22.328439 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:22.328439 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:22.328439 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:22.328439 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:22.328439 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:22.328439 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:22.328439 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:22.328439 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:22.328439 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:22.328439 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:22.328439 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:22.328439 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:22.328439 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:22.328439 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:22.328439 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:22.328439 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:22.331171 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:22.331171 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:22.331171 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:22.331171 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:22.331171 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:22.331171 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:22.331171 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:22.331171 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:22.331171 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:22.331171 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:22.331171 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:40:22.331171 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:22.331171 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 221/448] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:40:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:22.339038 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433d9a5-6230-4a93-84dd-6ee9f88b2fab None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:22.427852 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc63f18-7b17-419b-94a1-b55b77edc6c5 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:22.487766 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0589fb44-6da1-4257-a6ef-28e21fed2ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:22.488994 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589fb44-6da1-4257-a6ef-28e21fed2ca2 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:22.489469 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589fb44-6da1-4257-a6ef-28e21fed2ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:22.489934 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0589fb44-6da1-4257-a6ef-28e21fed2ca2 None admin] PATH_INFO: `/v3/groups/490d3612ce324804bb32d642c6b92f03` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:22.491230 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0589fb44-6da1-4257-a6ef-28e21fed2ca2 None admin] RBAC: Authorizing `identity:get_group(group_id=490d3612ce324804bb32d642c6b92f03)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:22.492696 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0589fb44-6da1-4257-a6ef-28e21fed2ca2 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:22.499386 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:22.499615 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 228/449] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Jul 17 16:40:22 2022] GET /identity/v3/groups/490d3612ce324804bb32d642c6b92f03 => generated 226 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:22.500407 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0589fb44-6da1-4257-a6ef-28e21fed2ca2 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:22.511345 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433d9a5-6230-4a93-84dd-6ee9f88b2fab None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:22.569349 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc7520-af4e-4c24-a3cb-6b87b9d76ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:22.570417 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc7520-af4e-4c24-a3cb-6b87b9d76ddd None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:22.570888 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc7520-af4e-4c24-a3cb-6b87b9d76ddd None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:22.571177 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc7520-af4e-4c24-a3cb-6b87b9d76ddd None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:22.571953 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc7520-af4e-4c24-a3cb-6b87b9d76ddd None admin] RBAC: Adding query filter params (name=admin) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:22.572265 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc7520-af4e-4c24-a3cb-6b87b9d76ddd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:22.573723 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc7520-af4e-4c24-a3cb-6b87b9d76ddd None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:22.585039 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 222/450] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:40:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:22.590995 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc7520-af4e-4c24-a3cb-6b87b9d76ddd None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:22.621941 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:22.636739 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0589fb44-6da1-4257-a6ef-28e21fed2ca2 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:22.684214 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f38c8e0-0b08-45cb-b5c9-e4aa65b9b958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:22.685072 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38c8e0-0b08-45cb-b5c9-e4aa65b9b958 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:22.685448 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38c8e0-0b08-45cb-b5c9-e4aa65b9b958 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:22.685795 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38c8e0-0b08-45cb-b5c9-e4aa65b9b958 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:22.686714 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f38c8e0-0b08-45cb-b5c9-e4aa65b9b958 None admin] RBAC: Adding query filter params (group.id=490d3612ce324804bb32d642c6b92f03, role.id=71cf67eb04704663aff719f6e3804f1a, scope.project.id=6704fc6d36e84a078282ac2057784505) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:22.687094 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f38c8e0-0b08-45cb-b5c9-e4aa65b9b958 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:22.689804 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f38c8e0-0b08-45cb-b5c9-e4aa65b9b958 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:22.707440 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 229/451] 10.222.0.40 () {56 vars in 1376 bytes} [Sun Jul 17 16:40:22 2022] GET /identity/v3/role_assignments?group.id=490d3612ce324804bb32d642c6b92f03&scope.project.id=6704fc6d36e84a078282ac2057784505&role.id=71cf67eb04704663aff719f6e3804f1a => generated 611 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:22.727223 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc7520-af4e-4c24-a3cb-6b87b9d76ddd None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:22.782319 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02daa17-7f4f-451e-bb2e-9b3e18c41669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:22.783616 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02daa17-7f4f-451e-bb2e-9b3e18c41669 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:22.784010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02daa17-7f4f-451e-bb2e-9b3e18c41669 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:22.784573 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02daa17-7f4f-451e-bb2e-9b3e18c41669 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:22.836245 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:22.893042 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02daa17-7f4f-451e-bb2e-9b3e18c41669 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:22.894686 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02daa17-7f4f-451e-bb2e-9b3e18c41669 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:23.022854 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 223/452] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:40:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 17 16:40:23.029390 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f38c8e0-0b08-45cb-b5c9-e4aa65b9b958 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:23.178576 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f38c8e0-0b08-45cb-b5c9-e4aa65b9b958 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:23.236431 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f490e-0643-4663-b4b2-613a9c9665bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:23.238091 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f490e-0643-4663-b4b2-613a9c9665bb None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:23.238596 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f490e-0643-4663-b4b2-613a9c9665bb None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:23.239086 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f490e-0643-4663-b4b2-613a9c9665bb None admin] PATH_INFO: `/v3/projects/3b2ec585f5d6476389a82d18a223a808` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:23.241162 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f490e-0643-4663-b4b2-613a9c9665bb None admin] RBAC: Authorizing `identity:get_project(project_id=3b2ec585f5d6476389a82d18a223a808)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:23.243836 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f490e-0643-4663-b4b2-613a9c9665bb None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:23.247239 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 230/453] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:40:23 2022] GET /identity/v3/projects/3b2ec585f5d6476389a82d18a223a808 => generated 344 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:23.253625 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02daa17-7f4f-451e-bb2e-9b3e18c41669 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:23.317162 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:23.388252 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02daa17-7f4f-451e-bb2e-9b3e18c41669 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:23.446716 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f894e66-186b-4eac-946d-a852dda8ad6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:23.448143 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f894e66-186b-4eac-946d-a852dda8ad6c None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:23.448654 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f894e66-186b-4eac-946d-a852dda8ad6c None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:23.449199 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f894e66-186b-4eac-946d-a852dda8ad6c None admin] PATH_INFO: `/v3/groups/5ce458bca7f34105b7709b4e4447bb29` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:23.451193 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f894e66-186b-4eac-946d-a852dda8ad6c None admin] RBAC: Authorizing `identity:get_group(group_id=5ce458bca7f34105b7709b4e4447bb29)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:23.453908 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f894e66-186b-4eac-946d-a852dda8ad6c None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:23.457833 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 224/454] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Jul 17 16:40:23 2022] GET /identity/v3/groups/5ce458bca7f34105b7709b4e4447bb29 => generated 229 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:23.464451 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f490e-0643-4663-b4b2-613a9c9665bb None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:23.500530 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:23.587563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f490e-0643-4663-b4b2-613a9c9665bb None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:23.623036 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ad8e6-a5dc-4ce7-8193-aadf8f0d056c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:23.623703 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ad8e6-a5dc-4ce7-8193-aadf8f0d056c None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:23.623995 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ad8e6-a5dc-4ce7-8193-aadf8f0d056c None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:23.624297 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ad8e6-a5dc-4ce7-8193-aadf8f0d056c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:23.625156 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ad8e6-a5dc-4ce7-8193-aadf8f0d056c None admin] RBAC: Adding query filter params (group.id=5ce458bca7f34105b7709b4e4447bb29, role.id=1025051975f64f689effa58dc4259e36, scope.project.id=3b2ec585f5d6476389a82d18a223a808) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:23.625465 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ad8e6-a5dc-4ce7-8193-aadf8f0d056c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:23.627654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ad8e6-a5dc-4ce7-8193-aadf8f0d056c None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:23.634069 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 231/455] 10.222.0.40 () {56 vars in 1376 bytes} [Sun Jul 17 16:40:23 2022] GET /identity/v3/role_assignments?group.id=5ce458bca7f34105b7709b4e4447bb29&scope.project.id=3b2ec585f5d6476389a82d18a223a808&role.id=1025051975f64f689effa58dc4259e36 => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:26.135187 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a51a7a-e088-4a64-a0f3-5643b6d8ddd1 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:26.136323 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a51a7a-e088-4a64-a0f3-5643b6d8ddd1 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:26.137006 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a51a7a-e088-4a64-a0f3-5643b6d8ddd1 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:26.138788 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 225/456] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:26.145529 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea313c96-7718-4cc8-892f-bc5935bc535d None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:26.146041 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea313c96-7718-4cc8-892f-bc5935bc535d None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:26.146115 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea313c96-7718-4cc8-892f-bc5935bc535d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:26.168834 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-ea313c96-7718-4cc8-892f-bc5935bc535d None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:26.170193 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-ea313c96-7718-4cc8-892f-bc5935bc535d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:26.283238 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 232/457] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 17 16:40:26.325698 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a51a7a-e088-4a64-a0f3-5643b6d8ddd1 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:26.358746 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37624c4-e86f-4753-836d-a7bd5aa176a0 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:26.359406 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37624c4-e86f-4753-836d-a7bd5aa176a0 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:26.360154 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37624c4-e86f-4753-836d-a7bd5aa176a0 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:26.363705 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 233/458] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:40:26.369345 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af01b8-c649-445d-9c5d-7e59b544eae6 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:26.369962 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af01b8-c649-445d-9c5d-7e59b544eae6 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:26.370478 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af01b8-c649-445d-9c5d-7e59b544eae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:26.402034 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-68af01b8-c649-445d-9c5d-7e59b544eae6 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:26.403201 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-68af01b8-c649-445d-9c5d-7e59b544eae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:26.425189 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:26.549776 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 234/459] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:26.550547 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a51a7a-e088-4a64-a0f3-5643b6d8ddd1 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:26.598969 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af01b8-c649-445d-9c5d-7e59b544eae6 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:26.604484 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:26.628418 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bcd8f6-8a57-4a77-af10-eb94a3d91e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:26.629279 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bcd8f6-8a57-4a77-af10-eb94a3d91e35 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:26.629569 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bcd8f6-8a57-4a77-af10-eb94a3d91e35 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:26.629850 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bcd8f6-8a57-4a77-af10-eb94a3d91e35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:26.654175 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bcd8f6-8a57-4a77-af10-eb94a3d91e35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:26.656043 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bcd8f6-8a57-4a77-af10-eb94a3d91e35 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:26.692817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-90bcd8f6-8a57-4a77-af10-eb94a3d91e35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:26.692817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:26.692817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:26.692817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:26.692817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:26.692817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:26.692817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:26.692817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:26.692817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:26.692817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:26.692817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:26.692817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:26.692817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:26.692817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:26.692817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:26.693819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:26.693819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:26.693819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:26.693819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:26.693819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:26.693819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:26.693819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:26.693819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:26.693819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:26.693819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:26.693819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:26.693819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:26.693819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:26.693819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:26.693819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:26.693819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:26.693819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:26.694648 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:26.694648 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:26.694648 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:26.694648 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:26.706564 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 226/460] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:26.713277 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bcd8f6-8a57-4a77-af10-eb94a3d91e35 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:26.805142 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af01b8-c649-445d-9c5d-7e59b544eae6 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:26.857602 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c7cec-00c2-48ae-a423-5b94cd7f1b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:26.858774 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c7cec-00c2-48ae-a423-5b94cd7f1b67 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:26.859245 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c7cec-00c2-48ae-a423-5b94cd7f1b67 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:26.859708 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c7cec-00c2-48ae-a423-5b94cd7f1b67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:26.869451 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bcd8f6-8a57-4a77-af10-eb94a3d91e35 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:26.883867 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c7cec-00c2-48ae-a423-5b94cd7f1b67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:26.885780 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c7cec-00c2-48ae-a423-5b94cd7f1b67 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:26.891629 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:26.919322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-af3c7cec-00c2-48ae-a423-5b94cd7f1b67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:26.919322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:26.919322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:26.919322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:26.919322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:26.919322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:26.919322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:26.919322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:26.919322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:26.919322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:26.919322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:26.919322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:26.919322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:26.919322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:26.919322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:26.920201 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:26.920201 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:26.920201 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:26.920201 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:26.920201 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:26.920201 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:26.920201 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:26.920201 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:26.920201 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:26.920201 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:26.920201 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:26.920201 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:26.920201 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:26.920201 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:26.920201 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:26.920201 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:26.920201 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:26.921553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:26.921553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:26.921553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:26.921553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:26.924184 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 235/461] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:40:26.928835 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f8cc6-1c31-46c0-8583-ed878a80025d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:26.928835 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f8cc6-1c31-46c0-8583-ed878a80025d None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:26.928835 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f8cc6-1c31-46c0-8583-ed878a80025d None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:26.928835 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f8cc6-1c31-46c0-8583-ed878a80025d None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:26.929190 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f8cc6-1c31-46c0-8583-ed878a80025d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:26.929190 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f8cc6-1c31-46c0-8583-ed878a80025d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:26.929190 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f8cc6-1c31-46c0-8583-ed878a80025d None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:26.930547 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c7cec-00c2-48ae-a423-5b94cd7f1b67 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:26.949037 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 227/462] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:40:26.950492 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f8cc6-1c31-46c0-8583-ed878a80025d None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:27.000094 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:27.099759 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f8cc6-1c31-46c0-8583-ed878a80025d None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:27.140088 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c7cec-00c2-48ae-a423-5b94cd7f1b67 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:27.184116 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b196cba8-8015-46ec-97ad-945d72536dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:27.185219 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196cba8-8015-46ec-97ad-945d72536dba None admin] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:27.185646 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196cba8-8015-46ec-97ad-945d72536dba None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:27.186054 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196cba8-8015-46ec-97ad-945d72536dba None admin] PATH_INFO: `/v3/users` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:27.186941 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b196cba8-8015-46ec-97ad-945d72536dba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:27.188581 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b196cba8-8015-46ec-97ad-945d72536dba None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:27.222185 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18249437-c24e-4bfc-9fa7-9d3476b8eab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:27.223746 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18249437-c24e-4bfc-9fa7-9d3476b8eab5 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:27.224282 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18249437-c24e-4bfc-9fa7-9d3476b8eab5 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:27.224864 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18249437-c24e-4bfc-9fa7-9d3476b8eab5 None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:27.225916 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18249437-c24e-4bfc-9fa7-9d3476b8eab5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:27.226586 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18249437-c24e-4bfc-9fa7-9d3476b8eab5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:27.228504 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18249437-c24e-4bfc-9fa7-9d3476b8eab5 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:27.244602 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 236/463] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:27.247396 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858661f0-ae46-437e-98a2-93ca11e0c000 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:27.248142 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858661f0-ae46-437e-98a2-93ca11e0c000 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:27.248863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858661f0-ae46-437e-98a2-93ca11e0c000 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:27.252770 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 237/464] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:27.254717 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858661f0-ae46-437e-98a2-93ca11e0c000 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:27.333288 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 228/465] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 17 16:40:26 2022] POST /identity/v3/users => generated 246 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 17 16:40:27.338132 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe21e2c-626c-4d22-a635-813e81343681 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:27.341662 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe21e2c-626c-4d22-a635-813e81343681 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:27.342531 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe21e2c-626c-4d22-a635-813e81343681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:27.382354 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-afe21e2c-626c-4d22-a635-813e81343681 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:27.384632 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-afe21e2c-626c-4d22-a635-813e81343681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:27.396159 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:27.516228 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858661f0-ae46-437e-98a2-93ca11e0c000 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:27.537819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 229/466] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:27.539419 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e555c-bcc6-4e5e-aa60-e1c573e0271c None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:27.539833 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e555c-bcc6-4e5e-aa60-e1c573e0271c None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:27.540270 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e555c-bcc6-4e5e-aa60-e1c573e0271c None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:27.545666 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 230/467] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:27.555849 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce287f-eb99-450d-aa8d-b880b5efc6b6 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:27.556610 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce287f-eb99-450d-aa8d-b880b5efc6b6 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:27.557335 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce287f-eb99-450d-aa8d-b880b5efc6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:27.588139 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8236f-7a90-4cb5-bf78-fbf8f5cedd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:27.589424 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8236f-7a90-4cb5-bf78-fbf8f5cedd4e None admin] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:27.591708 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8236f-7a90-4cb5-bf78-fbf8f5cedd4e None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:27.591708 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8236f-7a90-4cb5-bf78-fbf8f5cedd4e None admin] PATH_INFO: `/v3/users` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:27.591983 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-83ce287f-eb99-450d-aa8d-b880b5efc6b6 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:27.592110 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a8236f-7a90-4cb5-bf78-fbf8f5cedd4e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:27.592449 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-83ce287f-eb99-450d-aa8d-b880b5efc6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:27.594810 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a8236f-7a90-4cb5-bf78-fbf8f5cedd4e None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:27.624171 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:27.721901 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 231/468] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 17 16:40:27.723480 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce287f-eb99-450d-aa8d-b880b5efc6b6 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:27.776214 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 238/469] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 17 16:40:27 2022] POST /identity/v3/users => generated 248 bytes in 522 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 17 16:40:27.780992 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8236f-7a90-4cb5-bf78-fbf8f5cedd4e None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:27.951005 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce287f-eb99-450d-aa8d-b880b5efc6b6 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:28.004386 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f430f5-cd88-4acf-ad19-27a676ffa880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:28.005265 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f430f5-cd88-4acf-ad19-27a676ffa880 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:28.005559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f430f5-cd88-4acf-ad19-27a676ffa880 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:28.005841 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f430f5-cd88-4acf-ad19-27a676ffa880 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:28.036156 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f430f5-cd88-4acf-ad19-27a676ffa880 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:28.037680 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f430f5-cd88-4acf-ad19-27a676ffa880 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:28.077677 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8236f-7a90-4cb5-bf78-fbf8f5cedd4e None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:28.081537 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-64f430f5-cd88-4acf-ad19-27a676ffa880 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:28.081537 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:28.081537 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:28.081537 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:28.081537 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:28.081537 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:28.081537 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:28.081537 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:28.081537 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:28.081537 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:28.081537 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:28.081537 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:28.081537 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:28.081537 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:28.081537 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:28.082986 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:28.082986 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:28.082986 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:28.082986 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:28.082986 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:28.082986 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:28.082986 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:28.082986 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:28.082986 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:28.082986 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:28.082986 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:28.082986 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:28.082986 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:28.082986 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:28.082986 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:28.082986 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:28.082986 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:28.084620 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:28.084620 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:28.084620 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:28.084620 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:28.088402 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 232/470] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:28.099011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f430f5-cd88-4acf-ad19-27a676ffa880 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:28.106804 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:28.147454 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8426231c-45ee-4b32-9720-a98b31feb886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:28.148753 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8426231c-45ee-4b32-9720-a98b31feb886 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:28.149269 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8426231c-45ee-4b32-9720-a98b31feb886 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:28.150274 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8426231c-45ee-4b32-9720-a98b31feb886 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:28.158319 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:28.170892 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8426231c-45ee-4b32-9720-a98b31feb886 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:28.175677 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8426231c-45ee-4b32-9720-a98b31feb886 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:28.211410 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-8426231c-45ee-4b32-9720-a98b31feb886 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:28.211410 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:28.211410 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:28.211410 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:28.211410 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:28.211410 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:28.211410 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:28.211410 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:28.211410 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:28.211410 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:28.211410 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:28.211410 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:28.211410 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:28.211410 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:28.211410 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:28.213134 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:28.213134 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:28.213134 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:28.213134 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:28.213134 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:28.213134 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:28.213134 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:28.213134 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:28.213134 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:28.213134 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:28.213134 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:28.213134 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:28.213134 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:28.213134 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:28.213134 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:28.213134 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:28.213134 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:28.214878 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:28.214878 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:28.214878 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:28.214878 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:28.217353 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 239/471] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 440 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:28.227178 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8426231c-45ee-4b32-9720-a98b31feb886 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:28.331517 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f430f5-cd88-4acf-ad19-27a676ffa880 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:28.403964 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa4da8-82e5-4011-8796-2403e045af0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:28.405492 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa4da8-82e5-4011-8796-2403e045af0e None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:28.406221 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa4da8-82e5-4011-8796-2403e045af0e None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:28.407003 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa4da8-82e5-4011-8796-2403e045af0e None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:28.408884 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa4da8-82e5-4011-8796-2403e045af0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:28.409742 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa4da8-82e5-4011-8796-2403e045af0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:28.412668 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa4da8-82e5-4011-8796-2403e045af0e None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:28.437903 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 233/472] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:28.441733 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754143d6-cbf4-462f-9904-7b3e2b36bda0 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:28.442651 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754143d6-cbf4-462f-9904-7b3e2b36bda0 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:28.443481 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754143d6-cbf4-462f-9904-7b3e2b36bda0 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:28.446819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 234/473] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:28.448571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754143d6-cbf4-462f-9904-7b3e2b36bda0 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:28.456442 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8426231c-45ee-4b32-9720-a98b31feb886 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:28.519117 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e951f30-d235-4b29-899f-1f94f91c6df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:28.520236 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e951f30-d235-4b29-899f-1f94f91c6df0 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:28.520823 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e951f30-d235-4b29-899f-1f94f91c6df0 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:28.522610 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e951f30-d235-4b29-899f-1f94f91c6df0 None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:28.523070 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e951f30-d235-4b29-899f-1f94f91c6df0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:28.523737 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e951f30-d235-4b29-899f-1f94f91c6df0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:28.526617 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e951f30-d235-4b29-899f-1f94f91c6df0 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:28.557234 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 240/474] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:40:28.559814 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28af559-92ef-453d-b116-8669d9857393 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:28.560456 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28af559-92ef-453d-b116-8669d9857393 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:28.561101 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28af559-92ef-453d-b116-8669d9857393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:28.595775 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-f28af559-92ef-453d-b116-8669d9857393 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:28.597410 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-f28af559-92ef-453d-b116-8669d9857393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:28.617326 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:28.649297 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:28.660832 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754143d6-cbf4-462f-9904-7b3e2b36bda0 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:28.736915 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 241/475] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:28.736915 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28af559-92ef-453d-b116-8669d9857393 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:28.739633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643dcc5-790a-4465-9ca6-c650c3dbb975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:28.741104 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643dcc5-790a-4465-9ca6-c650c3dbb975 None admin] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:28.741849 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643dcc5-790a-4465-9ca6-c650c3dbb975 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:28.742579 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643dcc5-790a-4465-9ca6-c650c3dbb975 None admin] PATH_INFO: `/v3/users` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:28.744088 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c643dcc5-790a-4465-9ca6-c650c3dbb975 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:28.746943 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c643dcc5-790a-4465-9ca6-c650c3dbb975 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:28.877377 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28af559-92ef-453d-b116-8669d9857393 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:28.889427 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 235/476] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 17 16:40:28 2022] POST /identity/v3/users => generated 248 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:40:28.891610 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643dcc5-790a-4465-9ca6-c650c3dbb975 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:28.927137 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29665050-1c8e-4687-8a73-c1afa62ab897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:28.927973 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29665050-1c8e-4687-8a73-c1afa62ab897 None admin] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:28.928266 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29665050-1c8e-4687-8a73-c1afa62ab897 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:28.928545 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29665050-1c8e-4687-8a73-c1afa62ab897 None admin] PATH_INFO: `/v3/users` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:28.929313 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29665050-1c8e-4687-8a73-c1afa62ab897 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:28.930842 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29665050-1c8e-4687-8a73-c1afa62ab897 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:29.014185 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 242/477] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 17 16:40:28 2022] POST /identity/v3/users => generated 249 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:40:29.138137 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643dcc5-790a-4465-9ca6-c650c3dbb975 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:29.209399 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:29.229585 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed9149b-9b98-4d6e-8b72-c80633f469f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:29.230686 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9149b-9b98-4d6e-8b72-c80633f469f6 None admin] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:29.231171 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9149b-9b98-4d6e-8b72-c80633f469f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:29.231661 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9149b-9b98-4d6e-8b72-c80633f469f6 None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:29.233027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed9149b-9b98-4d6e-8b72-c80633f469f6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:29.236460 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed9149b-9b98-4d6e-8b72-c80633f469f6 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:29.278655 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.sql.core [None req-bed9149b-9b98-4d6e-8b72-c80633f469f6 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 17 16:40:29.278655 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 17 16:40:29.278655 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [parameters: {'id': '376e55dd5be34643948e85b05a40bb30', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 17 16:40:29.278655 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87199) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 17 16:40:29.308732 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-bed9149b-9b98-4d6e-8b72-c80633f469f6 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 17 16:40:29.308732 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:29.308732 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 17 16:40:29.308732 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 17 16:40:29.308732 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 17 16:40:29.308732 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 17 16:40:29.308732 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 17 16:40:29.308732 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application result = self._query(query) Jul 17 16:40:29.308732 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 17 16:40:29.308732 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application conn.query(q) Jul 17 16:40:29.308732 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 17 16:40:29.308732 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 17 16:40:29.308732 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 17 16:40:29.308732 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application result.read() Jul 17 16:40:29.308732 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 17 16:40:29.310459 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 17 16:40:29.310459 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 17 16:40:29.310459 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 17 16:40:29.310459 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 17 16:40:29.310459 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 17 16:40:29.310459 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 17 16:40:29.310459 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 17 16:40:29.310459 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 17 16:40:29.310459 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:29.310459 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 17 16:40:29.310459 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:29.310459 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:29.310459 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 17 16:40:29.310459 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 17 16:40:29.310459 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 17 16:40:29.310459 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return ref.to_dict() Jul 17 16:40:29.310459 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 17 16:40:29.310459 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application next(self.gen) Jul 17 16:40:29.312170 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 17 16:40:29.312170 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application yield resource Jul 17 16:40:29.312170 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 17 16:40:29.312170 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application next(self.gen) Jul 17 16:40:29.312170 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 17 16:40:29.312170 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self.session.rollback() Jul 17 16:40:29.312170 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 17 16:40:29.312170 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self.force_reraise() Jul 17 16:40:29.312170 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 17 16:40:29.312170 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise self.value Jul 17 16:40:29.312170 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 17 16:40:29.312170 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 17 16:40:29.312170 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 17 16:40:29.312170 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application session.commit() Jul 17 16:40:29.312170 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 17 16:40:29.312170 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 17 16:40:29.312170 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 17 16:40:29.315743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self._prepare_impl() Jul 17 16:40:29.315743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 17 16:40:29.315743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self.session.flush() Jul 17 16:40:29.315743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 17 16:40:29.315743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self._flush(objects) Jul 17 16:40:29.315743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 17 16:40:29.315743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 17 16:40:29.315743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 17 16:40:29.315743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application compat.raise_( Jul 17 16:40:29.315743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 17 16:40:29.315743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception Jul 17 16:40:29.315743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 17 16:40:29.315743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application flush_context.execute() Jul 17 16:40:29.315743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 17 16:40:29.315743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rec.execute(self) Jul 17 16:40:29.315743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 17 16:40:29.315743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 17 16:40:29.317491 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 17 16:40:29.317491 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 17 16:40:29.317491 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 17 16:40:29.317491 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 17 16:40:29.317491 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 17 16:40:29.317491 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 17 16:40:29.317491 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 17 16:40:29.317491 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 17 16:40:29.317491 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 17 16:40:29.317491 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 17 16:40:29.317491 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 17 16:40:29.317491 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 17 16:40:29.317491 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 17 16:40:29.317491 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 17 16:40:29.317491 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 17 16:40:29.317491 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception Jul 17 16:40:29.319031 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 17 16:40:29.319031 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 17 16:40:29.319031 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 17 16:40:29.319031 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 17 16:40:29.319031 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 17 16:40:29.319031 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application result = self._query(query) Jul 17 16:40:29.319031 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 17 16:40:29.319031 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application conn.query(q) Jul 17 16:40:29.319031 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 17 16:40:29.319031 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 17 16:40:29.319031 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 17 16:40:29.319031 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application result.read() Jul 17 16:40:29.319031 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 17 16:40:29.319031 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 17 16:40:29.319031 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 17 16:40:29.319031 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 17 16:40:29.323033 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 17 16:40:29.323033 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 17 16:40:29.323033 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 17 16:40:29.323033 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 17 16:40:29.323033 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 17 16:40:29.323033 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 17 16:40:29.323033 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [parameters: {'id': '376e55dd5be34643948e85b05a40bb30', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 17 16:40:29.323033 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 17 16:40:29.323033 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:29.323033 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 17 16:40:29.323033 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:29.323033 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:29.323033 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:29.323033 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:29.323033 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:29.323033 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:29.329920 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:29.329920 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:29.329920 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:29.329920 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:29.329920 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:29.329920 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:29.329920 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 17 16:40:29.329920 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 17 16:40:29.329920 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:29.329920 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:29.329920 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 17 16:40:29.329920 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 17 16:40:29.329920 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 17 16:40:29.329920 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 17 16:40:29.329920 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 17 16:40:29.329920 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:29.331457 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 236/478] 10.222.0.40 () {60 vars in 1131 bytes} [Sun Jul 17 16:40:28 2022] POST /identity/v3/roles => generated 144 bytes in 421 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:29.331576 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29665050-1c8e-4687-8a73-c1afa62ab897 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:29.359784 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:29.544680 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29665050-1c8e-4687-8a73-c1afa62ab897 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:29.610004 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f268f-ac01-4ba9-912e-c97bd3b0947d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:29.612007 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f268f-ac01-4ba9-912e-c97bd3b0947d None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:29.612514 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f268f-ac01-4ba9-912e-c97bd3b0947d None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:29.613042 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f268f-ac01-4ba9-912e-c97bd3b0947d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:29.635408 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f268f-ac01-4ba9-912e-c97bd3b0947d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:29.652512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-b91f268f-ac01-4ba9-912e-c97bd3b0947d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 17 16:40:29.652512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:29.652512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:29.652512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:29.652512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:29.652512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:29.652512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:29.652512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:29.652512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:29.652512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:29.652512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:29.652512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:29.652512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:29.652512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:29.652512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:29.652512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:29.654749 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:29.654749 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:29.654749 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:29.654749 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:29.654749 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:29.654749 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:29.654749 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:29.654749 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:29.654749 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:29.654749 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:29.654749 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:29.654749 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:29.654749 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:29.654749 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:29.654749 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:29.654749 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:29.656802 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:29.656802 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:29.656802 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:29.656802 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:29.656802 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:29.656802 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:29.656802 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:29.656802 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:29.656802 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:29.656802 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:29.656802 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 17 16:40:29.656802 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:29.656802 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 243/479] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 17 16:40:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:29.673468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed9149b-9b98-4d6e-8b72-c80633f469f6 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:29.915294 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed9149b-9b98-4d6e-8b72-c80633f469f6 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:29.986604 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1f7f1f-7308-4b66-bb37-6ff252cdad4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:29.987873 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f7f1f-7308-4b66-bb37-6ff252cdad4a None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:29.988382 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f7f1f-7308-4b66-bb37-6ff252cdad4a None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:29.988914 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1f7f1f-7308-4b66-bb37-6ff252cdad4a None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:29.990360 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1f7f1f-7308-4b66-bb37-6ff252cdad4a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:29.990926 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1f7f1f-7308-4b66-bb37-6ff252cdad4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:29.993417 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1f7f1f-7308-4b66-bb37-6ff252cdad4a None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:30.013151 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 237/480] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:40:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:30.580343 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f8c05-9802-4eee-bcdd-cde31211250a None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:30.580905 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f8c05-9802-4eee-bcdd-cde31211250a None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:30.581399 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f8c05-9802-4eee-bcdd-cde31211250a None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:30.584486 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 244/481] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:30.590523 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577e6d1-9d1d-410f-ab19-416825e5819c None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:30.594267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577e6d1-9d1d-410f-ab19-416825e5819c None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:30.594267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577e6d1-9d1d-410f-ab19-416825e5819c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:30.617310 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-b577e6d1-9d1d-410f-ab19-416825e5819c None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:30.618893 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-b577e6d1-9d1d-410f-ab19-416825e5819c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:30.693016 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:30.721210 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 238/482] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:30.769216 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f8c05-9802-4eee-bcdd-cde31211250a None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:30.852518 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:30.902616 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f8c05-9802-4eee-bcdd-cde31211250a None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:30.923822 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627c0fc-199d-47e8-98bd-d0ff895e7929 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:30.924301 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627c0fc-199d-47e8-98bd-d0ff895e7929 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:30.924814 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627c0fc-199d-47e8-98bd-d0ff895e7929 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:30.933559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 239/483] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:30 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:30.942216 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27cf55c-ab12-4df4-a353-034ea4806c61 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:30.942722 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27cf55c-ab12-4df4-a353-034ea4806c61 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:30.943220 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27cf55c-ab12-4df4-a353-034ea4806c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:30.949257 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d7e5b4-61a4-4c86-a36f-8a85bb17479f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:30.950081 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7e5b4-61a4-4c86-a36f-8a85bb17479f None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:30.950370 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7e5b4-61a4-4c86-a36f-8a85bb17479f None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:30.950651 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d7e5b4-61a4-4c86-a36f-8a85bb17479f None admin] PATH_INFO: `/v3/roles/service` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:30.962747 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d7e5b4-61a4-4c86-a36f-8a85bb17479f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:30.975379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-35d7e5b4-61a4-4c86-a36f-8a85bb17479f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:40:30.975379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:30.975379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:30.975379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:30.975379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:30.975379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:30.975379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:30.975379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:30.975379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:30.975379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:30.975379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:30.975379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:30.975379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:30.975379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:30.975379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:30.975379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:30.976334 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:30.976334 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:30.976334 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:30.976334 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:30.976334 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:30.976334 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:30.976334 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:30.976334 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:30.976334 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:30.976334 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:30.976334 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:30.976334 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:30.976334 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:30.976334 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:30.976334 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:30.976334 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:30.977167 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:30.977167 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:30.977167 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:30.977167 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:30.977167 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:30.977167 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:30.977167 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:30.977167 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:30.977167 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:30.977167 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:30.977167 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:40:30.977167 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:30.977167 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 245/484] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 17 16:40:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:30.982734 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d7e5b4-61a4-4c86-a36f-8a85bb17479f None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:30.992387 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-d27cf55c-ab12-4df4-a353-034ea4806c61 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:30.993654 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-d27cf55c-ab12-4df4-a353-034ea4806c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:31.127007 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 240/485] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:31.206566 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d7e5b4-61a4-4c86-a36f-8a85bb17479f None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:31.213280 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27cf55c-ab12-4df4-a353-034ea4806c61 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:31.317212 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:31.342606 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938007f-dc51-4f16-b102-4b2db76c59f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:31.343780 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938007f-dc51-4f16-b102-4b2db76c59f2 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:31.344299 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938007f-dc51-4f16-b102-4b2db76c59f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:31.344899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938007f-dc51-4f16-b102-4b2db76c59f2 None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:31.347170 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a938007f-dc51-4f16-b102-4b2db76c59f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:31.347739 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a938007f-dc51-4f16-b102-4b2db76c59f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:31.350422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a938007f-dc51-4f16-b102-4b2db76c59f2 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:31.368069 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 246/486] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 17 16:40:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:40:31.376847 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938007f-dc51-4f16-b102-4b2db76c59f2 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:31.549356 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27cf55c-ab12-4df4-a353-034ea4806c61 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:31.586222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:31.682074 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9607108-2f3e-4639-95a1-d817fb0e0038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:31.683391 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9607108-2f3e-4639-95a1-d817fb0e0038 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:31.683906 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9607108-2f3e-4639-95a1-d817fb0e0038 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:31.684387 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9607108-2f3e-4639-95a1-d817fb0e0038 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:31.726677 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9607108-2f3e-4639-95a1-d817fb0e0038 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:31.753533 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-c9607108-2f3e-4639-95a1-d817fb0e0038 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:40:31.753533 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:31.753533 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:31.753533 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:31.753533 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:31.753533 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:31.753533 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:31.753533 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:31.753533 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:31.753533 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:31.753533 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:31.753533 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:31.753533 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:31.753533 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:31.753533 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:31.753533 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:31.755221 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:31.755221 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:31.755221 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:31.755221 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:31.755221 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:31.755221 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:31.755221 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:31.755221 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:31.755221 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:31.755221 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:31.755221 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:31.755221 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:31.755221 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:31.755221 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:31.755221 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:31.755221 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:31.756743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:31.756743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:31.756743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:31.756743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:31.756743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:31.756743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:31.756743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:31.756743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:31.756743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:31.756743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:31.756743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:40:31.756743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:31.757915 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938007f-dc51-4f16-b102-4b2db76c59f2 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:31.762430 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 241/487] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 17 16:40:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 550 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:31.774263 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9607108-2f3e-4639-95a1-d817fb0e0038 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:31.823190 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa2a49-b787-4e70-86e3-ad950acbfe78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:31.824234 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa2a49-b787-4e70-86e3-ad950acbfe78 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:31.824662 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa2a49-b787-4e70-86e3-ad950acbfe78 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:31.825144 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa2a49-b787-4e70-86e3-ad950acbfe78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:31.840321 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aa2a49-b787-4e70-86e3-ad950acbfe78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:31.842417 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aa2a49-b787-4e70-86e3-ad950acbfe78 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:31.866920 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-c4aa2a49-b787-4e70-86e3-ad950acbfe78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:31.866920 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:31.866920 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:31.866920 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:31.866920 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:31.866920 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:31.866920 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:31.866920 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:31.866920 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:31.866920 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:31.866920 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:31.866920 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:31.866920 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:31.866920 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:31.866920 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:31.867888 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:31.867888 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:31.867888 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:31.867888 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:31.867888 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:31.867888 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:31.867888 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:31.867888 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:31.867888 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:31.867888 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:31.867888 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:31.867888 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:31.867888 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:31.867888 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:31.867888 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:31.867888 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:31.867888 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:31.868732 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:31.868732 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:31.868732 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:31.868732 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:31.871597 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 247/488] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 498 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:40:31.873864 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7120b1c9-c115-4438-98c4-877ccbddd121 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:31.874326 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7120b1c9-c115-4438-98c4-877ccbddd121 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:31.874747 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7120b1c9-c115-4438-98c4-877ccbddd121 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:31.877414 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 248/489] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:31.882413 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7120b1c9-c115-4438-98c4-877ccbddd121 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:31.971707 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9607108-2f3e-4639-95a1-d817fb0e0038 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:31.978190 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:32.021512 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce1983-c6b9-4be1-99c4-17281a49b269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:32.022397 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce1983-c6b9-4be1-99c4-17281a49b269 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:32.022780 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce1983-c6b9-4be1-99c4-17281a49b269 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:32.023304 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce1983-c6b9-4be1-99c4-17281a49b269 None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:32.024306 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce1983-c6b9-4be1-99c4-17281a49b269 None admin] RBAC: Adding query filter params (name=service) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:32.024712 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce1983-c6b9-4be1-99c4-17281a49b269 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:32.026235 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce1983-c6b9-4be1-99c4-17281a49b269 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:32.044822 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 242/490] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 17 16:40:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:32.044822 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43fb7a-7e50-4caa-9c92-e59d01caa77f None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:32.044822 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43fb7a-7e50-4caa-9c92-e59d01caa77f None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:32.044822 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43fb7a-7e50-4caa-9c92-e59d01caa77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:32.071291 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-9e43fb7a-7e50-4caa-9c92-e59d01caa77f None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:32.073962 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-9e43fb7a-7e50-4caa-9c92-e59d01caa77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:32.104270 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7120b1c9-c115-4438-98c4-877ccbddd121 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:32.118438 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:32.187375 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d856536d-d6b2-4d9e-b515-a2008da628cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:32.188612 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d856536d-d6b2-4d9e-b515-a2008da628cd None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:32.189266 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d856536d-d6b2-4d9e-b515-a2008da628cd None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:32.189836 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d856536d-d6b2-4d9e-b515-a2008da628cd None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:32.191251 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d856536d-d6b2-4d9e-b515-a2008da628cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:32.191902 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d856536d-d6b2-4d9e-b515-a2008da628cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:32.196046 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d856536d-d6b2-4d9e-b515-a2008da628cd None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:32.213126 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 243/491] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:32.215129 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e43fb7a-7e50-4caa-9c92-e59d01caa77f None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:32.216224 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 249/492] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:32.224834 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d856536d-d6b2-4d9e-b515-a2008da628cd None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:32.356539 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d856536d-d6b2-4d9e-b515-a2008da628cd None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:32.359507 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:32.396231 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ae6e0-38ce-4c95-9546-b24bf4ebe24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:32.397080 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ae6e0-38ce-4c95-9546-b24bf4ebe24e None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:32.397369 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ae6e0-38ce-4c95-9546-b24bf4ebe24e None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:32.397684 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ae6e0-38ce-4c95-9546-b24bf4ebe24e None admin] PATH_INFO: `/v3/users/nova` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:32.401241 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e43fb7a-7e50-4caa-9c92-e59d01caa77f None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:32.406868 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ae6e0-38ce-4c95-9546-b24bf4ebe24e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:32.407457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ae6e0-38ce-4c95-9546-b24bf4ebe24e None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:32.424837 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-184ae6e0-38ce-4c95-9546-b24bf4ebe24e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 17 16:40:32.424837 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:32.424837 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:32.424837 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:32.424837 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:32.424837 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:32.424837 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:32.424837 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:32.424837 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:32.424837 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:32.424837 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:32.424837 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:32.424837 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:40:32.424837 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:40:32.424837 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:40:32.424837 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:40:32.426670 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:32.426670 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:32.426670 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:40:32.426670 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:40:32.426670 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:40:32.426670 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:40:32.426670 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:32.426670 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:32.426670 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:32.426670 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:32.426670 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:32.426670 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:32.426670 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:32.426670 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:32.426670 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:32.426670 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:32.426670 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:32.426670 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:32.428715 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:32.428715 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:32.428715 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:40:32.428715 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:40:32.428715 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:40:32.428715 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:40:32.428715 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:40:32.428715 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:40:32.428715 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 17 16:40:32.428715 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:32.431092 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 250/493] 10.222.0.40 () {56 vars in 1093 bytes} [Sun Jul 17 16:40:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:32.432186 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ae6e0-38ce-4c95-9546-b24bf4ebe24e None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:32.477017 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f973e-a2da-4dbd-8a8a-29f91830484f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:32.478866 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f973e-a2da-4dbd-8a8a-29f91830484f None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:32.479577 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f973e-a2da-4dbd-8a8a-29f91830484f None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:32.480279 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589f973e-a2da-4dbd-8a8a-29f91830484f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:32.503914 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f973e-a2da-4dbd-8a8a-29f91830484f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:32.507133 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589f973e-a2da-4dbd-8a8a-29f91830484f None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:32.544683 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-589f973e-a2da-4dbd-8a8a-29f91830484f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:32.544683 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:32.544683 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:32.544683 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:32.544683 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:32.544683 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:32.544683 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:32.544683 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:32.544683 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:32.544683 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:32.544683 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:32.544683 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:32.544683 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:32.544683 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:32.544683 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:32.546598 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:32.546598 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:32.546598 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:32.546598 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:32.546598 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:32.546598 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:32.546598 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:32.546598 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:32.546598 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:32.546598 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:32.546598 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:32.546598 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:32.546598 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:32.546598 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:32.546598 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:32.546598 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:32.546598 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:32.548739 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:32.548739 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:32.548739 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:32.548739 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:32.549175 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ae6e0-38ce-4c95-9546-b24bf4ebe24e None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:32.552483 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 244/494] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:32.554300 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f973e-a2da-4dbd-8a8a-29f91830484f None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:32.587817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:32.589696 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14024dd8-f999-4411-8e4b-15b198639943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:32.590698 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14024dd8-f999-4411-8e4b-15b198639943 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:32.591200 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14024dd8-f999-4411-8e4b-15b198639943 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:32.591658 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14024dd8-f999-4411-8e4b-15b198639943 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:32.604451 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14024dd8-f999-4411-8e4b-15b198639943 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:32.612971 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-14024dd8-f999-4411-8e4b-15b198639943 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:40:32.612971 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:32.612971 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:32.612971 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:32.612971 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:32.612971 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:32.612971 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:32.612971 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:32.612971 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:32.612971 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:32.612971 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:32.612971 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:32.612971 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:32.612971 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:32.612971 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:32.612971 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:32.613896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:32.613896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:32.613896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:32.613896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:32.613896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:32.613896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:32.613896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:32.613896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:32.613896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:32.613896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:32.613896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:32.613896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:32.613896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:32.613896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:32.613896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:32.613896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:32.614811 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:32.614811 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:32.614811 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:32.614811 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:32.614811 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:32.614811 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:32.614811 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:32.614811 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:32.614811 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:32.614811 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:32.614811 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:40:32.614811 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:32.622403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 251/495] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 17 16:40:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:32.629897 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14024dd8-f999-4411-8e4b-15b198639943 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:32.653596 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:32.719660 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589f973e-a2da-4dbd-8a8a-29f91830484f None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:32.759605 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bcf47d-b49f-4387-9edc-71f4ead57126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:32.760536 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcf47d-b49f-4387-9edc-71f4ead57126 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:32.760706 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcf47d-b49f-4387-9edc-71f4ead57126 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:32.760982 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcf47d-b49f-4387-9edc-71f4ead57126 None admin] PATH_INFO: `/v3/users` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:32.761987 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bcf47d-b49f-4387-9edc-71f4ead57126 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:32.762371 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bcf47d-b49f-4387-9edc-71f4ead57126 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:32.763939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bcf47d-b49f-4387-9edc-71f4ead57126 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:32.806803 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14024dd8-f999-4411-8e4b-15b198639943 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:32.819919 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 245/496] 10.222.0.40 () {56 vars in 1133 bytes} [Sun Jul 17 16:40:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:32.819919 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bcf47d-b49f-4387-9edc-71f4ead57126 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:32.847250 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f3ff7-2e49-4558-bec9-425f25019107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:32.848027 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f3ff7-2e49-4558-bec9-425f25019107 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:32.848393 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f3ff7-2e49-4558-bec9-425f25019107 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:32.848741 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f3ff7-2e49-4558-bec9-425f25019107 None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:32.849540 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42f3ff7-2e49-4558-bec9-425f25019107 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:32.849855 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42f3ff7-2e49-4558-bec9-425f25019107 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:32.851269 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42f3ff7-2e49-4558-bec9-425f25019107 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:32.864448 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 252/497] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:32.865751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be0540-eb57-4fc9-bec3-7d0e838960e0 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:32.866038 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be0540-eb57-4fc9-bec3-7d0e838960e0 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:32.866321 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be0540-eb57-4fc9-bec3-7d0e838960e0 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:32.867587 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 253/498] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:32.868318 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be0540-eb57-4fc9-bec3-7d0e838960e0 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:32.940662 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:32.982711 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bcf47d-b49f-4387-9edc-71f4ead57126 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:32.990483 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:33.018888 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be0540-eb57-4fc9-bec3-7d0e838960e0 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:33.029508 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d74e8a7-5a68-48e1-9a92-7c1f392a4959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:33.030621 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74e8a7-5a68-48e1-9a92-7c1f392a4959 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:33.031102 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74e8a7-5a68-48e1-9a92-7c1f392a4959 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:33.031551 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74e8a7-5a68-48e1-9a92-7c1f392a4959 None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:33.032973 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d74e8a7-5a68-48e1-9a92-7c1f392a4959 None admin] RBAC: Adding query filter params (name=service) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:33.033484 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d74e8a7-5a68-48e1-9a92-7c1f392a4959 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:33.035740 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d74e8a7-5a68-48e1-9a92-7c1f392a4959 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:33.050367 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 246/499] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 17 16:40:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:33.052813 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4b784-9fa1-4d31-a3f4-2d5b1d0ede75 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:33.053327 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4b784-9fa1-4d31-a3f4-2d5b1d0ede75 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:33.053821 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4b784-9fa1-4d31-a3f4-2d5b1d0ede75 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:33.056830 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 247/500] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:33.057889 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4b784-9fa1-4d31-a3f4-2d5b1d0ede75 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:33.076747 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76cfba-c156-41ae-b6be-279f285b1eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:33.077492 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76cfba-c156-41ae-b6be-279f285b1eb3 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:33.077778 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76cfba-c156-41ae-b6be-279f285b1eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:33.078061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a76cfba-c156-41ae-b6be-279f285b1eb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:33.152829 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a76cfba-c156-41ae-b6be-279f285b1eb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:33.154238 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a76cfba-c156-41ae-b6be-279f285b1eb3 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:33.190970 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4b784-9fa1-4d31-a3f4-2d5b1d0ede75 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:33.230166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:33.236063 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595be2a1-2db1-479c-9eef-45fbde0d92d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:33.237140 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595be2a1-2db1-479c-9eef-45fbde0d92d7 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:33.237427 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595be2a1-2db1-479c-9eef-45fbde0d92d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:33.237709 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595be2a1-2db1-479c-9eef-45fbde0d92d7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:33.246093 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595be2a1-2db1-479c-9eef-45fbde0d92d7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:33.247572 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595be2a1-2db1-479c-9eef-45fbde0d92d7 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:33.266499 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:33.269815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-595be2a1-2db1-479c-9eef-45fbde0d92d7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 17 16:40:33.269815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:33.269815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:33.269815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:33.269815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:33.269815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:33.269815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:33.269815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:33.269815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:33.269815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:33.269815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:33.269815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:33.269815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:40:33.269815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:40:33.269815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:40:33.269815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:40:33.270886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:33.270886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:33.270886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:40:33.270886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:40:33.270886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:40:33.270886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:40:33.270886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:33.270886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:33.270886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:33.270886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:33.270886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:33.270886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:33.270886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:33.270886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:33.270886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:33.270886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:33.270886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:33.270886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:33.271875 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:33.271875 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:33.271875 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:40:33.271875 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:40:33.271875 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:40:33.271875 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:40:33.271875 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:40:33.271875 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:40:33.271875 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 17 16:40:33.271875 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:33.271875 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 248/501] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:40:33 2022] GET /identity/v3/users/glance => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:33.272834 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282afdc-e63a-4553-a171-a15392e64cf8 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:33.272834 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282afdc-e63a-4553-a171-a15392e64cf8 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:33.273035 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282afdc-e63a-4553-a171-a15392e64cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:33.289255 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 254/502] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:40:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 17 16:40:33.290132 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76cfba-c156-41ae-b6be-279f285b1eb3 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:33.292018 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-b282afdc-e63a-4553-a171-a15392e64cf8 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:33.293309 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-b282afdc-e63a-4553-a171-a15392e64cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:33.368753 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 249/503] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:33.370106 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767702ed-3c9e-45a3-b228-dc5bab4b27ff None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:33.370461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767702ed-3c9e-45a3-b228-dc5bab4b27ff None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:33.370647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767702ed-3c9e-45a3-b228-dc5bab4b27ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:33.396364 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-767702ed-3c9e-45a3-b228-dc5bab4b27ff None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:33.398441 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-767702ed-3c9e-45a3-b228-dc5bab4b27ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:33.450976 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a76cfba-c156-41ae-b6be-279f285b1eb3 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:33.487147 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 250/504] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:33.488312 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767702ed-3c9e-45a3-b228-dc5bab4b27ff None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:33.499320 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e1043-6350-41ef-9319-2c2d169c29f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:33.500126 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e1043-6350-41ef-9319-2c2d169c29f8 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:33.500411 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e1043-6350-41ef-9319-2c2d169c29f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:33.500686 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121e1043-6350-41ef-9319-2c2d169c29f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:33.512277 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121e1043-6350-41ef-9319-2c2d169c29f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:33.513785 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121e1043-6350-41ef-9319-2c2d169c29f8 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:33.537686 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-121e1043-6350-41ef-9319-2c2d169c29f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:33.537686 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:33.537686 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:33.537686 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:33.537686 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:33.537686 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:33.537686 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:33.537686 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:33.537686 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:33.537686 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:33.537686 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:33.537686 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:33.537686 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:33.537686 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:33.537686 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:33.541871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:33.541871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:33.541871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:33.541871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:33.541871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:33.541871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:33.541871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:33.541871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:33.541871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:33.541871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:33.541871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:33.541871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:33.541871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:33.541871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:33.541871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:33.541871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:33.541871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:33.548768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:33.548768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:33.548768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:33.548768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:33.548768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 255/505] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:33.548768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e1043-6350-41ef-9319-2c2d169c29f8 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:33.573044 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:33.582211 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:33.648924 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767702ed-3c9e-45a3-b228-dc5bab4b27ff None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:33.654492 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121e1043-6350-41ef-9319-2c2d169c29f8 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:33.692113 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2999f5-c206-4af8-a5e8-7ad3cea13e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:33.692844 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2999f5-c206-4af8-a5e8-7ad3cea13e9f None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:33.693127 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2999f5-c206-4af8-a5e8-7ad3cea13e9f None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:33.693404 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2999f5-c206-4af8-a5e8-7ad3cea13e9f None admin] PATH_INFO: `/v3/users` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:33.694280 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2999f5-c206-4af8-a5e8-7ad3cea13e9f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:33.694847 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2999f5-c206-4af8-a5e8-7ad3cea13e9f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:33.696228 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b0cc0-d1cc-44fa-82af-70396500d3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:33.696382 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2999f5-c206-4af8-a5e8-7ad3cea13e9f None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:33.697246 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b0cc0-d1cc-44fa-82af-70396500d3cd None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:33.697635 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b0cc0-d1cc-44fa-82af-70396500d3cd None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:33.697966 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b0cc0-d1cc-44fa-82af-70396500d3cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:33.710563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b0cc0-d1cc-44fa-82af-70396500d3cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:33.712024 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b0cc0-d1cc-44fa-82af-70396500d3cd None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:33.718146 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 251/506] 10.222.0.40 () {56 vars in 1137 bytes} [Sun Jul 17 16:40:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:33.719028 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2999f5-c206-4af8-a5e8-7ad3cea13e9f None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:33.739528 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-769b0cc0-d1cc-44fa-82af-70396500d3cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:33.739528 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:33.739528 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:33.739528 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:33.739528 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:33.739528 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:33.739528 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:33.739528 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:33.739528 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:33.739528 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:33.739528 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:33.739528 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:33.739528 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:33.739528 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:33.739528 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:33.740460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:33.740460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:33.740460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:33.740460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:33.740460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:33.740460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:33.740460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:33.740460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:33.740460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:33.740460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:33.740460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:33.740460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:33.740460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:33.740460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:33.740460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:33.740460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:33.740460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:33.741362 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:33.741362 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:33.741362 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:33.741362 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:33.741577 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 256/507] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:33.742864 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b0cc0-d1cc-44fa-82af-70396500d3cd None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:33.886364 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2999f5-c206-4af8-a5e8-7ad3cea13e9f None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:33.894041 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:33.906309 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:33.913321 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b0cc0-d1cc-44fa-82af-70396500d3cd None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:33.936112 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd050802-1c98-45cf-9535-553e169c5813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:33.937113 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd050802-1c98-45cf-9535-553e169c5813 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:33.937485 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd050802-1c98-45cf-9535-553e169c5813 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:33.937852 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd050802-1c98-45cf-9535-553e169c5813 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:33.953137 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd050802-1c98-45cf-9535-553e169c5813 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:33.957478 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c600756-55f6-4c07-a577-ee60df722047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:33.958295 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c600756-55f6-4c07-a577-ee60df722047 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:33.958573 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c600756-55f6-4c07-a577-ee60df722047 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:33.958839 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c600756-55f6-4c07-a577-ee60df722047 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:33.963553 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-fd050802-1c98-45cf-9535-553e169c5813 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:40:33.963553 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:33.963553 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:33.963553 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:33.963553 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:33.963553 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:33.963553 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:33.963553 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:33.963553 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:33.963553 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:33.963553 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:33.963553 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:33.963553 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:33.963553 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:33.963553 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:33.963553 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:33.965215 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:33.965215 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:33.965215 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:33.965215 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:33.965215 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:33.965215 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:33.965215 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:33.965215 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:33.965215 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:33.965215 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:33.965215 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:33.965215 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:33.965215 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:33.965215 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:33.965215 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:33.965215 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:33.966697 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:33.966697 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:33.966697 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:33.966697 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:33.966697 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:33.966697 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:33.966697 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:33.966697 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:33.966697 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:33.966697 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:33.966697 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:40:33.966697 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:33.966697 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 252/508] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 17 16:40:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:33.966697 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd050802-1c98-45cf-9535-553e169c5813 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:33.975838 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c600756-55f6-4c07-a577-ee60df722047 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:33.977508 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c600756-55f6-4c07-a577-ee60df722047 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:33.997124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-7c600756-55f6-4c07-a577-ee60df722047 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:33.997124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:33.997124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:33.997124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:33.997124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:33.997124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:33.997124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:33.997124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:33.997124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:33.997124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:33.997124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:33.997124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:33.997124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:33.997124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:33.997124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:33.998768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:33.998768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:33.998768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:33.998768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:33.998768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:33.998768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:33.998768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:33.998768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:33.998768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:33.998768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:33.998768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:33.998768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:33.998768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:33.998768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:33.998768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:33.998768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:33.998768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:34.000426 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:34.000426 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:34.000426 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:34.000426 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:34.000426 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 257/509] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:34.000426 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c600756-55f6-4c07-a577-ee60df722047 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:34.133776 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd050802-1c98-45cf-9535-553e169c5813 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:34.146527 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c600756-55f6-4c07-a577-ee60df722047 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:34.188326 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e93ad9-c4c7-46bc-a3cc-089b1dffa110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:34.189214 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b38eeb-b47b-48b7-8052-1a4019366236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:34.189601 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e93ad9-c4c7-46bc-a3cc-089b1dffa110 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:34.189817 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b38eeb-b47b-48b7-8052-1a4019366236 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:34.189923 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e93ad9-c4c7-46bc-a3cc-089b1dffa110 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:34.190168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e93ad9-c4c7-46bc-a3cc-089b1dffa110 None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:34.190315 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b38eeb-b47b-48b7-8052-1a4019366236 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:34.190602 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b38eeb-b47b-48b7-8052-1a4019366236 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:34.190919 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e93ad9-c4c7-46bc-a3cc-089b1dffa110 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:34.191234 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e93ad9-c4c7-46bc-a3cc-089b1dffa110 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:34.192649 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e93ad9-c4c7-46bc-a3cc-089b1dffa110 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:34.204516 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:34.214208 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 253/510] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:34.214208 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e93ad9-c4c7-46bc-a3cc-089b1dffa110 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:34.260865 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:34.287362 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b38eeb-b47b-48b7-8052-1a4019366236 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:34.289936 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b38eeb-b47b-48b7-8052-1a4019366236 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:34.355006 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e93ad9-c4c7-46bc-a3cc-089b1dffa110 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:34.393065 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1ae7b-69c2-4ecc-aea6-9f50f3f7a81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:34.393744 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1ae7b-69c2-4ecc-aea6-9f50f3f7a81a None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:34.394032 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1ae7b-69c2-4ecc-aea6-9f50f3f7a81a None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:34.394312 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1ae7b-69c2-4ecc-aea6-9f50f3f7a81a None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:34.395057 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae1ae7b-69c2-4ecc-aea6-9f50f3f7a81a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:34.395370 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae1ae7b-69c2-4ecc-aea6-9f50f3f7a81a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:34.396779 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae1ae7b-69c2-4ecc-aea6-9f50f3f7a81a None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:34.412839 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 254/511] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:34.413913 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1ae7b-69c2-4ecc-aea6-9f50f3f7a81a None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:34.432722 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 258/512] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:40:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 17 16:40:34.434139 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b38eeb-b47b-48b7-8052-1a4019366236 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:34.494332 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:34.539366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:34.552606 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1ae7b-69c2-4ecc-aea6-9f50f3f7a81a None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:34.569102 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b38eeb-b47b-48b7-8052-1a4019366236 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:34.607791 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d40b19-47dc-44f6-93f8-727aa25ad910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:34.608671 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d40b19-47dc-44f6-93f8-727aa25ad910 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:34.609067 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d40b19-47dc-44f6-93f8-727aa25ad910 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:34.609427 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d40b19-47dc-44f6-93f8-727aa25ad910 None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:34.610959 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d40b19-47dc-44f6-93f8-727aa25ad910 None admin] RBAC: Adding query filter params (name=service) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:34.611386 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d40b19-47dc-44f6-93f8-727aa25ad910 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:34.614233 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d40b19-47dc-44f6-93f8-727aa25ad910 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:34.622326 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe2bf9-2a5a-48be-b19f-7d3db703aba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:34.623158 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe2bf9-2a5a-48be-b19f-7d3db703aba5 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:34.623513 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe2bf9-2a5a-48be-b19f-7d3db703aba5 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:34.623942 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe2bf9-2a5a-48be-b19f-7d3db703aba5 None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:34.624825 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe2bf9-2a5a-48be-b19f-7d3db703aba5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:34.625215 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe2bf9-2a5a-48be-b19f-7d3db703aba5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:34.625391 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 255/513] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 17 16:40:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:34.627227 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d40b19-47dc-44f6-93f8-727aa25ad910 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:34.627829 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe2bf9-2a5a-48be-b19f-7d3db703aba5 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:34.646839 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 259/514] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:34.648239 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe2bf9-2a5a-48be-b19f-7d3db703aba5 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:34.768945 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d40b19-47dc-44f6-93f8-727aa25ad910 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:34.776069 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:34.781246 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe2bf9-2a5a-48be-b19f-7d3db703aba5 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:34.821485 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42635a-f6e4-475c-b0b9-fd03b093ffe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:34.822937 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42635a-f6e4-475c-b0b9-fd03b093ffe4 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:34.823398 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42635a-f6e4-475c-b0b9-fd03b093ffe4 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:34.823886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42635a-f6e4-475c-b0b9-fd03b093ffe4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:34.827031 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ddb53b-aaf2-454f-b2b8-8f172addf63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:34.827978 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddb53b-aaf2-454f-b2b8-8f172addf63a None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:34.828316 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddb53b-aaf2-454f-b2b8-8f172addf63a None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:34.828673 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ddb53b-aaf2-454f-b2b8-8f172addf63a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:34.837046 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ddb53b-aaf2-454f-b2b8-8f172addf63a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:34.838576 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ddb53b-aaf2-454f-b2b8-8f172addf63a None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:34.839165 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f42635a-f6e4-475c-b0b9-fd03b093ffe4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:34.841892 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f42635a-f6e4-475c-b0b9-fd03b093ffe4 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:34.854462 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-e0ddb53b-aaf2-454f-b2b8-8f172addf63a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:40:34.854462 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:34.854462 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:34.854462 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:34.854462 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:34.854462 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:34.854462 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:34.854462 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:34.854462 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:34.854462 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:34.854462 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:34.854462 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:34.854462 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:40:34.854462 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:40:34.854462 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:40:34.856129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:40:34.856129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:34.856129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:34.856129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:34.856129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:34.856129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:34.856129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:34.856129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:34.856129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:34.856129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:34.856129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:34.856129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:34.856129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:34.856129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:34.856129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:34.856129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:34.856129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:34.857778 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:40:34.857778 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:40:34.857778 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:40:34.857778 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:40:34.857778 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:40:34.857778 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:40:34.857778 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:40:34.857778 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:34.857778 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 260/515] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:40:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:34.857778 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ddb53b-aaf2-454f-b2b8-8f172addf63a None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:34.868819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-4f42635a-f6e4-475c-b0b9-fd03b093ffe4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 17 16:40:34.868819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:34.868819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:34.868819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:34.868819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:34.868819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:34.868819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:34.868819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:34.868819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:34.868819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:34.868819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:34.868819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:34.868819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:40:34.868819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:40:34.868819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:40:34.868819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:40:34.870658 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:34.870658 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:34.870658 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:40:34.870658 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:40:34.870658 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:40:34.870658 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:40:34.870658 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:34.870658 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:34.870658 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:34.870658 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:34.870658 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:34.870658 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:34.870658 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:34.870658 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:34.870658 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:34.870658 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:34.870658 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:34.870658 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:34.872365 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:34.872365 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:34.872365 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:40:34.872365 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:40:34.872365 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:40:34.872365 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:40:34.872365 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:40:34.872365 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:40:34.872365 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 17 16:40:34.872365 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:34.872365 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 256/516] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 17 16:40:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:34.873507 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42635a-f6e4-475c-b0b9-fd03b093ffe4 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:34.896187 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:34.997755 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ddb53b-aaf2-454f-b2b8-8f172addf63a None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:35.007254 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42635a-f6e4-475c-b0b9-fd03b093ffe4 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:35.044854 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6546d8e-4b23-4ada-9465-f022dbe91e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:35.045598 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6546d8e-4b23-4ada-9465-f022dbe91e2f None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:35.045878 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6546d8e-4b23-4ada-9465-f022dbe91e2f None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:35.046160 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6546d8e-4b23-4ada-9465-f022dbe91e2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:35.058640 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6546d8e-4b23-4ada-9465-f022dbe91e2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:35.058997 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adaa6d0-6328-4df0-90fc-ddf38737dd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:35.060137 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6546d8e-4b23-4ada-9465-f022dbe91e2f None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:35.060323 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adaa6d0-6328-4df0-90fc-ddf38737dd3d None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:35.060830 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adaa6d0-6328-4df0-90fc-ddf38737dd3d None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:35.061323 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adaa6d0-6328-4df0-90fc-ddf38737dd3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:35.080403 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-b6546d8e-4b23-4ada-9465-f022dbe91e2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:35.080403 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:35.080403 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:35.080403 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:35.080403 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:35.080403 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:35.080403 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:35.080403 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:35.080403 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:35.080403 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:35.080403 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:35.080403 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:35.080403 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:35.080403 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:35.080403 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:35.082240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:35.082240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:35.082240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:35.082240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:35.082240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:35.082240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:35.082240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:35.082240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:35.082240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:35.082240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:35.082240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:35.082240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:35.082240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:35.082240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:35.082240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:35.082240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:35.082240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:35.084298 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:35.084298 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:35.084298 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:35.084298 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:35.084298 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 257/517] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:35.084298 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6546d8e-4b23-4ada-9465-f022dbe91e2f None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:35.090032 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adaa6d0-6328-4df0-90fc-ddf38737dd3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:35.092846 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adaa6d0-6328-4df0-90fc-ddf38737dd3d None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:35.128053 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-5adaa6d0-6328-4df0-90fc-ddf38737dd3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:35.128053 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:35.128053 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:35.128053 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:35.128053 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:35.128053 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:35.128053 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:35.128053 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:35.128053 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:35.128053 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:35.128053 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:35.128053 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:35.128053 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:35.128053 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:35.128053 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:35.129856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:35.129856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:35.129856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:35.129856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:35.129856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:35.129856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:35.129856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:35.129856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:35.129856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:35.129856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:35.129856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:35.129856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:35.129856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:35.129856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:35.129856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:35.129856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:35.129856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:35.131509 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:35.131509 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:35.131509 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:35.131509 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:35.131509 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 261/518] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:35.132018 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adaa6d0-6328-4df0-90fc-ddf38737dd3d None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:35.158655 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:35.163524 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:35.212579 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6546d8e-4b23-4ada-9465-f022dbe91e2f None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:35.262963 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbdae1a-f65e-499e-90a1-3ca6d37e9cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:35.263847 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbdae1a-f65e-499e-90a1-3ca6d37e9cc0 None admin] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:35.264188 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbdae1a-f65e-499e-90a1-3ca6d37e9cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:35.264516 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbdae1a-f65e-499e-90a1-3ca6d37e9cc0 None admin] PATH_INFO: `/v3/users` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:35.265360 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbdae1a-f65e-499e-90a1-3ca6d37e9cc0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:35.267919 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbdae1a-f65e-499e-90a1-3ca6d37e9cc0 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:35.281503 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adaa6d0-6328-4df0-90fc-ddf38737dd3d None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:35.325614 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70345ad-2d1f-4bb6-8329-611f47f56868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:35.326247 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70345ad-2d1f-4bb6-8329-611f47f56868 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:35.326547 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70345ad-2d1f-4bb6-8329-611f47f56868 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:35.326830 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70345ad-2d1f-4bb6-8329-611f47f56868 None admin] PATH_INFO: `/v3/projects` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:35.327767 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70345ad-2d1f-4bb6-8329-611f47f56868 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:35.328165 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70345ad-2d1f-4bb6-8329-611f47f56868 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:35.330448 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70345ad-2d1f-4bb6-8329-611f47f56868 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:35.346185 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 258/519] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 17 16:40:35 2022] POST /identity/v3/users => generated 247 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:40:35.347081 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbdae1a-f65e-499e-90a1-3ca6d37e9cc0 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:35.362589 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 262/520] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:40:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:35.364102 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70345ad-2d1f-4bb6-8329-611f47f56868 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:35.500423 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:35.509343 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbdae1a-f65e-499e-90a1-3ca6d37e9cc0 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:35.537131 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:35.556981 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4321d-2159-4e82-ade5-b6228dad0ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:35.557687 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4321d-2159-4e82-ade5-b6228dad0ef5 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:35.557976 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4321d-2159-4e82-ade5-b6228dad0ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:35.558294 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4321d-2159-4e82-ade5-b6228dad0ef5 None admin] PATH_INFO: `/v3/users` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:35.559145 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4321d-2159-4e82-ade5-b6228dad0ef5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:35.559458 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4321d-2159-4e82-ade5-b6228dad0ef5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:35.560992 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f4321d-2159-4e82-ade5-b6228dad0ef5 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:35.561693 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70345ad-2d1f-4bb6-8329-611f47f56868 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:35.593219 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 259/521] 10.222.0.40 () {56 vars in 1139 bytes} [Sun Jul 17 16:40:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:35.595287 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4321d-2159-4e82-ade5-b6228dad0ef5 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:35.614938 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c469c-9016-4eb7-a6a4-1d942d221a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:35.615641 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c469c-9016-4eb7-a6a4-1d942d221a4e None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:35.615929 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c469c-9016-4eb7-a6a4-1d942d221a4e None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:35.616216 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c469c-9016-4eb7-a6a4-1d942d221a4e None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:35.616996 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5c469c-9016-4eb7-a6a4-1d942d221a4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:35.621597 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5c469c-9016-4eb7-a6a4-1d942d221a4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:35.621597 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5c469c-9016-4eb7-a6a4-1d942d221a4e None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:35.636161 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 263/522] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:35.637172 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c469c-9016-4eb7-a6a4-1d942d221a4e None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:35.791834 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4321d-2159-4e82-ade5-b6228dad0ef5 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:35.819248 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c469c-9016-4eb7-a6a4-1d942d221a4e None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:35.853011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c6ba7-9dc3-4ac1-b89b-a658777b0de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:35.853624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c6ba7-9dc3-4ac1-b89b-a658777b0de5 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:35.853909 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c6ba7-9dc3-4ac1-b89b-a658777b0de5 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:35.854191 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7c6ba7-9dc3-4ac1-b89b-a658777b0de5 None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:35.854932 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7c6ba7-9dc3-4ac1-b89b-a658777b0de5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:35.855327 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7c6ba7-9dc3-4ac1-b89b-a658777b0de5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:35.856753 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7c6ba7-9dc3-4ac1-b89b-a658777b0de5 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:35.870496 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:35.880100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 260/523] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:35.880100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c6ba7-9dc3-4ac1-b89b-a658777b0de5 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:35.889047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc8a80-54d7-4cfb-9d5b-f4933892b65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:35.890123 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc8a80-54d7-4cfb-9d5b-f4933892b65f None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:35.890563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc8a80-54d7-4cfb-9d5b-f4933892b65f None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:35.891018 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc8a80-54d7-4cfb-9d5b-f4933892b65f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:35.893538 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc8a80-54d7-4cfb-9d5b-f4933892b65f None admin] RBAC: Adding query filter params (role.id=adb9c27c6e5246fdb0cf386358a822cd, scope.project.id=927057f9a5af4833aeac9992203a223c, user.id=03244087f1a34bddb11ea46aba11a148) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:35.893538 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc8a80-54d7-4cfb-9d5b-f4933892b65f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:35.895644 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc8a80-54d7-4cfb-9d5b-f4933892b65f None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:35.904866 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 264/524] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:40:35 2022] GET /identity/v3/role_assignments?user.id=03244087f1a34bddb11ea46aba11a148&scope.project.id=927057f9a5af4833aeac9992203a223c&role.id=adb9c27c6e5246fdb0cf386358a822cd => generated 259 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:35.906200 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc8a80-54d7-4cfb-9d5b-f4933892b65f None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:35.971904 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:36.093500 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc8a80-54d7-4cfb-9d5b-f4933892b65f None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:36.094834 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7c6ba7-9dc3-4ac1-b89b-a658777b0de5 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:36.134131 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916afa51-78ec-4d48-b231-29a2806dca01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:36.134870 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916afa51-78ec-4d48-b231-29a2806dca01 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:36.135161 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916afa51-78ec-4d48-b231-29a2806dca01 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:36.135446 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916afa51-78ec-4d48-b231-29a2806dca01 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:36.146226 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916afa51-78ec-4d48-b231-29a2806dca01 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:36.147337 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916afa51-78ec-4d48-b231-29a2806dca01 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:36.163354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-916afa51-78ec-4d48-b231-29a2806dca01 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 17 16:40:36.163354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:36.163354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:36.163354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:36.163354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:36.163354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:36.163354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:36.163354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:36.163354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:36.163354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:36.163354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:36.163354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:36.163354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:40:36.163354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:40:36.163354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:40:36.163354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:40:36.165273 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:36.165273 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:36.165273 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:40:36.165273 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:40:36.165273 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:40:36.165273 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:40:36.165273 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:36.165273 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:36.165273 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:36.165273 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:36.165273 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:36.165273 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:36.165273 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:36.165273 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:36.165273 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:36.165273 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:36.165273 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:36.165273 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:36.166934 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:36.166934 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:36.166934 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:40:36.166934 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:40:36.166934 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:40:36.166934 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:40:36.166934 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:40:36.166934 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:40:36.166934 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 17 16:40:36.166934 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:36.166934 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 265/525] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:40:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:36.169014 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916afa51-78ec-4d48-b231-29a2806dca01 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:36.176616 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6843427-8dbe-4038-9d61-629cb4b9c1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:36.178307 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6843427-8dbe-4038-9d61-629cb4b9c1a7 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:36.179131 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6843427-8dbe-4038-9d61-629cb4b9c1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:36.179888 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6843427-8dbe-4038-9d61-629cb4b9c1a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:36.239722 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:36.291484 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916afa51-78ec-4d48-b231-29a2806dca01 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:36.331181 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6843427-8dbe-4038-9d61-629cb4b9c1a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:36.332989 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2af907-f7a3-4154-96c2-ca15d65c7eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:36.333458 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6843427-8dbe-4038-9d61-629cb4b9c1a7 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:36.334938 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2af907-f7a3-4154-96c2-ca15d65c7eff None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:36.336124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2af907-f7a3-4154-96c2-ca15d65c7eff None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:36.338365 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2af907-f7a3-4154-96c2-ca15d65c7eff None admin] PATH_INFO: `/v3/projects/service` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:36.356484 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2af907-f7a3-4154-96c2-ca15d65c7eff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:36.360013 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2af907-f7a3-4154-96c2-ca15d65c7eff None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:36.385773 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:36.400255 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-8a2af907-f7a3-4154-96c2-ca15d65c7eff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:40:36.400255 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:36.400255 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:36.400255 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:36.400255 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:36.400255 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:36.400255 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:36.400255 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:36.400255 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:36.400255 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:36.400255 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:36.400255 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:36.400255 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:40:36.400255 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:40:36.400255 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:40:36.406744 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:40:36.406744 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:36.406744 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:36.406744 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:36.406744 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:36.406744 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:36.406744 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:36.406744 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:36.406744 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:36.406744 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:36.406744 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:36.406744 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:36.406744 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:36.406744 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:36.406744 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:36.406744 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:36.406744 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:36.407758 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:40:36.407758 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:40:36.407758 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:40:36.407758 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:40:36.407758 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:40:36.407758 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:40:36.407758 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:40:36.407758 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:36.407758 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 266/526] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:40:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:36.408221 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2af907-f7a3-4154-96c2-ca15d65c7eff None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:36.473972 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 261/527] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:40:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 602 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 17 16:40:36.475269 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6843427-8dbe-4038-9d61-629cb4b9c1a7 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:36.590579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2af907-f7a3-4154-96c2-ca15d65c7eff None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:36.637531 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:36.675041 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043dc3a-4668-44d4-b3ab-c457ce99f50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:36.677153 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043dc3a-4668-44d4-b3ab-c457ce99f50f None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:36.677937 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043dc3a-4668-44d4-b3ab-c457ce99f50f None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:36.678697 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043dc3a-4668-44d4-b3ab-c457ce99f50f None admin] PATH_INFO: `/v3/users` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:36.680931 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5043dc3a-4668-44d4-b3ab-c457ce99f50f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:36.681580 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6843427-8dbe-4038-9d61-629cb4b9c1a7 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:36.682274 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5043dc3a-4668-44d4-b3ab-c457ce99f50f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:36.685452 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5043dc3a-4668-44d4-b3ab-c457ce99f50f None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:36.753346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5e2ee-4a5f-4965-bc81-fd824df10e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:36.755056 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5e2ee-4a5f-4965-bc81-fd824df10e19 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:36.755841 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5e2ee-4a5f-4965-bc81-fd824df10e19 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:36.756630 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5e2ee-4a5f-4965-bc81-fd824df10e19 None admin] PATH_INFO: `/v3/projects` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:36.758810 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5e2ee-4a5f-4965-bc81-fd824df10e19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:36.759763 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5e2ee-4a5f-4965-bc81-fd824df10e19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:36.763015 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5e2ee-4a5f-4965-bc81-fd824df10e19 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:36.770969 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 267/528] 10.222.0.40 () {56 vars in 1137 bytes} [Sun Jul 17 16:40:36 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:36.773131 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043dc3a-4668-44d4-b3ab-c457ce99f50f None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:36.787452 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:36.805086 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 262/529] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:40:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:36.807092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5e2ee-4a5f-4965-bc81-fd824df10e19 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:36.998004 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043dc3a-4668-44d4-b3ab-c457ce99f50f None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:37.032490 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5e2ee-4a5f-4965-bc81-fd824df10e19 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:37.078506 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c37109-bcd9-4c14-a5db-e2d6f17e1bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:37.080438 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c37109-bcd9-4c14-a5db-e2d6f17e1bde None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:37.080962 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c37109-bcd9-4c14-a5db-e2d6f17e1bde None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:37.082444 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c37109-bcd9-4c14-a5db-e2d6f17e1bde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:37.106076 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c37109-bcd9-4c14-a5db-e2d6f17e1bde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:37.107715 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc99b4-d9ae-45e6-9029-84056e3e39f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:37.108747 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c37109-bcd9-4c14-a5db-e2d6f17e1bde None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:37.108926 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc99b4-d9ae-45e6-9029-84056e3e39f6 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:37.109243 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc99b4-d9ae-45e6-9029-84056e3e39f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:37.109717 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc99b4-d9ae-45e6-9029-84056e3e39f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:37.148453 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-34c37109-bcd9-4c14-a5db-e2d6f17e1bde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:37.148453 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:37.148453 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:37.148453 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:37.148453 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:37.148453 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:37.148453 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:37.148453 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:37.148453 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:37.148453 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:37.148453 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:37.148453 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:37.148453 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:37.148453 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:37.148453 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:37.150155 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:37.150155 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:37.150155 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:37.150155 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:37.150155 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:37.150155 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:37.150155 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:37.150155 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:37.150155 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:37.150155 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:37.150155 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:37.150155 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:37.150155 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:37.150155 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:37.150155 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:37.150155 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:37.150155 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:37.151770 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:37.151770 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:37.151770 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:37.151770 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:37.151770 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 268/530] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:37.153202 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c37109-bcd9-4c14-a5db-e2d6f17e1bde None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:37.215631 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:37.254298 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:37.261967 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cc99b4-d9ae-45e6-9029-84056e3e39f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:37.267384 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cc99b4-d9ae-45e6-9029-84056e3e39f6 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:37.350268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c37109-bcd9-4c14-a5db-e2d6f17e1bde None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:37.413070 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a12b1-eef2-422a-87ec-c35e8fe592f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:37.414266 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a12b1-eef2-422a-87ec-c35e8fe592f9 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:37.414807 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a12b1-eef2-422a-87ec-c35e8fe592f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:37.415367 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a12b1-eef2-422a-87ec-c35e8fe592f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:37.417291 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581a12b1-eef2-422a-87ec-c35e8fe592f9 None admin] RBAC: Adding query filter params (role.id=adb9c27c6e5246fdb0cf386358a822cd, scope.project.id=927057f9a5af4833aeac9992203a223c, user.id=9e13bcc0f657445cbbdea5b78563f0d3) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:37.417979 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581a12b1-eef2-422a-87ec-c35e8fe592f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:37.421194 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581a12b1-eef2-422a-87ec-c35e8fe592f9 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:37.431859 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 269/531] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:40:37 2022] GET /identity/v3/role_assignments?user.id=9e13bcc0f657445cbbdea5b78563f0d3&scope.project.id=927057f9a5af4833aeac9992203a223c&role.id=adb9c27c6e5246fdb0cf386358a822cd => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:37.433087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a12b1-eef2-422a-87ec-c35e8fe592f9 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:37.486398 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 263/532] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:40:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 680 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 17 16:40:37.494973 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc99b4-d9ae-45e6-9029-84056e3e39f6 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:37.645636 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:37.679276 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a12b1-eef2-422a-87ec-c35e8fe592f9 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:37.753614 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca8c23-223d-4789-b4c6-76bfdc5cea3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:37.754844 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca8c23-223d-4789-b4c6-76bfdc5cea3b None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:37.755350 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca8c23-223d-4789-b4c6-76bfdc5cea3b None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:37.755805 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca8c23-223d-4789-b4c6-76bfdc5cea3b None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:37.757161 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edca8c23-223d-4789-b4c6-76bfdc5cea3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:37.757686 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edca8c23-223d-4789-b4c6-76bfdc5cea3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:37.760207 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edca8c23-223d-4789-b4c6-76bfdc5cea3b None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:37.784027 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 270/533] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:37.794434 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca8c23-223d-4789-b4c6-76bfdc5cea3b None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:37.821798 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:37.832611 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc99b4-d9ae-45e6-9029-84056e3e39f6 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:37.947605 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc82cd-b55c-42b6-ba0b-1ccc1f678741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:37.948831 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc82cd-b55c-42b6-ba0b-1ccc1f678741 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:37.949342 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc82cd-b55c-42b6-ba0b-1ccc1f678741 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:37.949839 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc82cd-b55c-42b6-ba0b-1ccc1f678741 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:37.993211 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc82cd-b55c-42b6-ba0b-1ccc1f678741 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:37.996302 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc82cd-b55c-42b6-ba0b-1ccc1f678741 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:38.047020 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-c0dc82cd-b55c-42b6-ba0b-1ccc1f678741 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:38.047020 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:38.047020 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:38.047020 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:38.047020 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:38.047020 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:38.047020 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:38.047020 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:38.047020 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:38.047020 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:38.047020 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:38.047020 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:38.047020 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:38.047020 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:38.047020 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:38.050037 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:38.050037 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:38.050037 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:38.050037 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:38.050037 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:38.050037 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:38.050037 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:38.050037 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:38.050037 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:38.050037 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:38.050037 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:38.050037 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:38.050037 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:38.050037 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:38.050037 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:38.050037 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:38.050037 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:38.051724 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:38.051724 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:38.051724 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:38.051724 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:38.051724 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 264/534] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 556 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:40:38.062438 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc82cd-b55c-42b6-ba0b-1ccc1f678741 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:38.136280 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca8c23-223d-4789-b4c6-76bfdc5cea3b None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:38.249592 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae5620-0fa6-4c49-b5cc-5bf0b1b1b644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:38.251443 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae5620-0fa6-4c49-b5cc-5bf0b1b1b644 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:38.251905 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae5620-0fa6-4c49-b5cc-5bf0b1b1b644 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:38.252363 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae5620-0fa6-4c49-b5cc-5bf0b1b1b644 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:38.267843 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ae5620-0fa6-4c49-b5cc-5bf0b1b1b644 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:38.270711 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ae5620-0fa6-4c49-b5cc-5bf0b1b1b644 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:38.295592 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc82cd-b55c-42b6-ba0b-1ccc1f678741 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:38.299207 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-d8ae5620-0fa6-4c49-b5cc-5bf0b1b1b644 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:40:38.299207 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:38.299207 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:38.299207 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:38.299207 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:38.299207 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:38.299207 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:38.299207 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:38.299207 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:38.299207 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:38.299207 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:38.299207 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:38.299207 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:40:38.299207 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:40:38.299207 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:40:38.300855 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:40:38.300855 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:38.300855 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:38.300855 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:38.300855 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:38.300855 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:38.300855 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:38.300855 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:38.300855 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:38.300855 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:38.300855 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:38.300855 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:38.300855 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:38.300855 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:38.300855 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:38.300855 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:38.300855 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:38.302673 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:40:38.302673 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:40:38.302673 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:40:38.302673 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:40:38.302673 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:40:38.302673 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:40:38.302673 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:40:38.302673 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:38.302673 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 271/535] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:40:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 510 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:38.304261 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391d6e1-83fa-4a1b-98ca-8c0f707b76e5 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:38.304825 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391d6e1-83fa-4a1b-98ca-8c0f707b76e5 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:38.305073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391d6e1-83fa-4a1b-98ca-8c0f707b76e5 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:38.313911 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 272/536] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:38.315611 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391d6e1-83fa-4a1b-98ca-8c0f707b76e5 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:38.344559 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:38.351573 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2296e-2314-4cba-bbdc-3e4160787d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:38.352261 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2296e-2314-4cba-bbdc-3e4160787d3b None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:38.352549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2296e-2314-4cba-bbdc-3e4160787d3b None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:38.352902 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2296e-2314-4cba-bbdc-3e4160787d3b None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:38.354329 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d2296e-2314-4cba-bbdc-3e4160787d3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:38.354650 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d2296e-2314-4cba-bbdc-3e4160787d3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:38.356385 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d2296e-2314-4cba-bbdc-3e4160787d3b None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:38.371377 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 265/537] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:38.373125 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e7767-ebd9-44fb-815e-89b000b52f03 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:38.373470 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e7767-ebd9-44fb-815e-89b000b52f03 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:38.373753 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e7767-ebd9-44fb-815e-89b000b52f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:38.395140 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-402e7767-ebd9-44fb-815e-89b000b52f03 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:38.396478 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-402e7767-ebd9-44fb-815e-89b000b52f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:38.401132 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:38.509571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 266/538] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:38.509571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e7767-ebd9-44fb-815e-89b000b52f03 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:38.534593 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391d6e1-83fa-4a1b-98ca-8c0f707b76e5 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:38.601876 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05cc58-b1b4-4743-bd36-d6f12a8b08fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:38.603621 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05cc58-b1b4-4743-bd36-d6f12a8b08fe None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:38.604144 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05cc58-b1b4-4743-bd36-d6f12a8b08fe None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:38.605308 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05cc58-b1b4-4743-bd36-d6f12a8b08fe None admin] PATH_INFO: `/v3/projects` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:38.607254 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05cc58-b1b4-4743-bd36-d6f12a8b08fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:38.607804 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05cc58-b1b4-4743-bd36-d6f12a8b08fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:38.611412 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05cc58-b1b4-4743-bd36-d6f12a8b08fe None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:38.640067 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 273/539] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:40:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:38.644858 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05cc58-b1b4-4743-bd36-d6f12a8b08fe None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:38.736171 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e7767-ebd9-44fb-815e-89b000b52f03 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:38.806645 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4ad3c9-b6c7-4ec1-a4c0-2b704314c7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:38.808053 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ad3c9-b6c7-4ec1-a4c0-2b704314c7e4 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:38.808559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ad3c9-b6c7-4ec1-a4c0-2b704314c7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:38.809485 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:38.809932 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ad3c9-b6c7-4ec1-a4c0-2b704314c7e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:38.824915 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4ad3c9-b6c7-4ec1-a4c0-2b704314c7e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:38.827670 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4ad3c9-b6c7-4ec1-a4c0-2b704314c7e4 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:38.833115 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:38.861559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-9f4ad3c9-b6c7-4ec1-a4c0-2b704314c7e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:40:38.861559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:38.861559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:38.861559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:38.861559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:38.861559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:38.861559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:38.861559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:38.861559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:38.861559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:38.861559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:38.861559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:38.861559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:40:38.861559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:40:38.861559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:40:38.863244 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:40:38.863244 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:38.863244 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:38.863244 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:38.863244 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:38.863244 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:38.863244 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:38.863244 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:38.863244 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:38.863244 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:38.863244 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:38.863244 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:38.863244 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:38.863244 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:38.863244 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:38.863244 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:38.863244 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:38.864928 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:40:38.864928 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:40:38.864928 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:40:38.864928 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:40:38.864928 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:40:38.864928 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:40:38.864928 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:40:38.864928 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:38.869655 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 267/540] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:40:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:38.871616 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4ad3c9-b6c7-4ec1-a4c0-2b704314c7e4 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:38.885145 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05cc58-b1b4-4743-bd36-d6f12a8b08fe None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:38.959614 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012876e1-6d05-4f9a-9263-f171e549d83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:38.961698 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012876e1-6d05-4f9a-9263-f171e549d83c None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:38.962618 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012876e1-6d05-4f9a-9263-f171e549d83c None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:38.963390 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012876e1-6d05-4f9a-9263-f171e549d83c None admin] PATH_INFO: `/v3/roles/service` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:38.986744 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012876e1-6d05-4f9a-9263-f171e549d83c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:39.005001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-012876e1-6d05-4f9a-9263-f171e549d83c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:40:39.005001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:39.005001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:39.005001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:39.005001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:39.005001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:39.005001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:39.005001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:39.005001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:39.005001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:39.005001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:39.005001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:39.005001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:39.005001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:39.005001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:39.005001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:39.006733 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:39.006733 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:39.006733 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:39.006733 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:39.006733 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:39.006733 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:39.006733 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:39.006733 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:39.006733 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:39.006733 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:39.006733 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:39.006733 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:39.006733 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:39.006733 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:39.006733 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:39.006733 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:39.008314 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:39.008314 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:39.008314 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:39.008314 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:39.008314 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:39.008314 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:39.008314 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:39.008314 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:39.008314 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:39.008314 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:39.008314 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:40:39.008314 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:39.018276 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 274/541] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 17 16:40:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:39.020493 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012876e1-6d05-4f9a-9263-f171e549d83c None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:39.085239 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4ad3c9-b6c7-4ec1-a4c0-2b704314c7e4 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:39.153355 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44447a-2870-4a63-b811-5008fd259e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:39.154743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44447a-2870-4a63-b811-5008fd259e76 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:39.155125 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44447a-2870-4a63-b811-5008fd259e76 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:39.155620 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44447a-2870-4a63-b811-5008fd259e76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:39.158187 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44447a-2870-4a63-b811-5008fd259e76 None admin] RBAC: Adding query filter params (role.id=adb9c27c6e5246fdb0cf386358a822cd, scope.project.id=927057f9a5af4833aeac9992203a223c, user.id=9663edc8502b4aecb5317b24e4a5816b) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:39.158909 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44447a-2870-4a63-b811-5008fd259e76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:39.161878 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44447a-2870-4a63-b811-5008fd259e76 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:39.180843 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 268/542] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:40:38 2022] GET /identity/v3/role_assignments?user.id=9663edc8502b4aecb5317b24e4a5816b&scope.project.id=927057f9a5af4833aeac9992203a223c&role.id=adb9c27c6e5246fdb0cf386358a822cd => generated 259 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:39.180843 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44447a-2870-4a63-b811-5008fd259e76 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:39.298768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012876e1-6d05-4f9a-9263-f171e549d83c None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:39.306421 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:39.398202 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44447a-2870-4a63-b811-5008fd259e76 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:39.426840 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:39.457609 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753cbb94-2ab5-4fdf-9c44-9ae1a0bf83ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:39.458878 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cbb94-2ab5-4fdf-9c44-9ae1a0bf83ce None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:39.458878 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cbb94-2ab5-4fdf-9c44-9ae1a0bf83ce None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:39.459193 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cbb94-2ab5-4fdf-9c44-9ae1a0bf83ce None admin] PATH_INFO: `/v3/projects` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:39.459763 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753cbb94-2ab5-4fdf-9c44-9ae1a0bf83ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:39.460081 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753cbb94-2ab5-4fdf-9c44-9ae1a0bf83ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:39.461667 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753cbb94-2ab5-4fdf-9c44-9ae1a0bf83ce None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:39.474029 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d62475-0ca2-4f94-acb8-87a6dc4aaadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:39.475656 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d62475-0ca2-4f94-acb8-87a6dc4aaadb None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:39.476428 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d62475-0ca2-4f94-acb8-87a6dc4aaadb None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:39.477177 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d62475-0ca2-4f94-acb8-87a6dc4aaadb None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:39.478947 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d62475-0ca2-4f94-acb8-87a6dc4aaadb None admin] RBAC: Adding query filter params (name=service) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:39.479716 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d62475-0ca2-4f94-acb8-87a6dc4aaadb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:39.482620 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d62475-0ca2-4f94-acb8-87a6dc4aaadb None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:39.495661 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 275/543] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:40:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 476 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:39.510360 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 269/544] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 17 16:40:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:39.510620 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753cbb94-2ab5-4fdf-9c44-9ae1a0bf83ce None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:39.518657 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d62475-0ca2-4f94-acb8-87a6dc4aaadb None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:39.752305 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d62475-0ca2-4f94-acb8-87a6dc4aaadb None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:39.768415 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753cbb94-2ab5-4fdf-9c44-9ae1a0bf83ce None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:39.805140 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:39.821445 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf32c55-2847-4cf2-8ab2-32fe541fee28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:39.825655 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf32c55-2847-4cf2-8ab2-32fe541fee28 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:39.825655 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf32c55-2847-4cf2-8ab2-32fe541fee28 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:39.825655 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf32c55-2847-4cf2-8ab2-32fe541fee28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:39.835818 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a1902-2a16-4098-8333-05d623672849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:39.837097 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a1902-2a16-4098-8333-05d623672849 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:39.837677 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a1902-2a16-4098-8333-05d623672849 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:39.838255 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a1902-2a16-4098-8333-05d623672849 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:39.840054 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a1902-2a16-4098-8333-05d623672849 None admin] RBAC: Adding query filter params (role.id=adb9c27c6e5246fdb0cf386358a822cd, scope.project.id=927057f9a5af4833aeac9992203a223c, user.id=0b7122fc744d446aa60b25844c68f15b) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:39.841515 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a1902-2a16-4098-8333-05d623672849 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:39.843965 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49a1902-2a16-4098-8333-05d623672849 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:39.850375 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf32c55-2847-4cf2-8ab2-32fe541fee28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:39.853878 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf32c55-2847-4cf2-8ab2-32fe541fee28 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:39.857663 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 276/545] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:40:39 2022] GET /identity/v3/role_assignments?user.id=0b7122fc744d446aa60b25844c68f15b&scope.project.id=927057f9a5af4833aeac9992203a223c&role.id=adb9c27c6e5246fdb0cf386358a822cd => generated 259 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:39.860951 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be1e15-d347-44f4-bd71-d92cfe6b5e22 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:39.861495 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be1e15-d347-44f4-bd71-d92cfe6b5e22 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:39.862012 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be1e15-d347-44f4-bd71-d92cfe6b5e22 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:39.864971 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 277/546] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:39.875439 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51cd4d-c758-4c2e-adc1-28e353ec5579 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:39.876244 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51cd4d-c758-4c2e-adc1-28e353ec5579 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:39.876650 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51cd4d-c758-4c2e-adc1-28e353ec5579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:39.893478 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-dcf32c55-2847-4cf2-8ab2-32fe541fee28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:39.893478 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:39.893478 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:39.893478 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:39.893478 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:39.893478 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:39.893478 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:39.893478 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:39.893478 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:39.893478 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:39.893478 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:39.893478 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:39.893478 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:39.893478 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:39.893478 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:39.895317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:39.895317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:39.895317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:39.895317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:39.895317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:39.895317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:39.895317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:39.895317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:39.895317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:39.895317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:39.895317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:39.895317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:39.895317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:39.895317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:39.895317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:39.895317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:39.895317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:39.897122 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:39.897122 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:39.897122 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:39.897122 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:39.901573 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 270/547] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:39.921697 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf32c55-2847-4cf2-8ab2-32fe541fee28 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:39.939446 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-da51cd4d-c758-4c2e-adc1-28e353ec5579 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:39.941403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-da51cd4d-c758-4c2e-adc1-28e353ec5579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:40.069120 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:40.114519 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 278/548] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:40.137597 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf32c55-2847-4cf2-8ab2-32fe541fee28 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:40.161456 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51cd4d-c758-4c2e-adc1-28e353ec5579 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:40.184978 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f94a6-4b7b-4e22-9931-c8aa8bba5c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:40.186067 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f94a6-4b7b-4e22-9931-c8aa8bba5c98 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:40.186519 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f94a6-4b7b-4e22-9931-c8aa8bba5c98 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:40.186974 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f94a6-4b7b-4e22-9931-c8aa8bba5c98 None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:40.188184 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2f94a6-4b7b-4e22-9931-c8aa8bba5c98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:40.188714 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2f94a6-4b7b-4e22-9931-c8aa8bba5c98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:40.191201 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2f94a6-4b7b-4e22-9931-c8aa8bba5c98 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:40.228848 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 271/549] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:40:40.236109 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f94a6-4b7b-4e22-9931-c8aa8bba5c98 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:40.368829 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:40.475998 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51cd4d-c758-4c2e-adc1-28e353ec5579 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:40.519979 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f94a6-4b7b-4e22-9931-c8aa8bba5c98 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:40.542868 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9809ad0f-846b-4ab8-bcc6-86b753d48244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:40.544396 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809ad0f-846b-4ab8-bcc6-86b753d48244 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:40.545000 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809ad0f-846b-4ab8-bcc6-86b753d48244 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:40.545495 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809ad0f-846b-4ab8-bcc6-86b753d48244 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:40.560403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9809ad0f-846b-4ab8-bcc6-86b753d48244 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:40.568768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-9809ad0f-846b-4ab8-bcc6-86b753d48244 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:40:40.568768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:40.568768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:40.568768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:40.568768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:40.568768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:40.568768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:40.568768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:40.568768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:40.568768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:40.568768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:40.568768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:40.568768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:40.568768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:40.568768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:40.568768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:40.570180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:40.570180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:40.570180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:40.570180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:40.570180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:40.570180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:40.570180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:40.570180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:40.570180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:40.570180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:40.570180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:40.570180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:40.570180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:40.570180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:40.570180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:40.570180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:40.571652 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:40.571652 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:40.571652 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:40.571652 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:40.571652 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:40.571652 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:40.571652 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:40.571652 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:40.571652 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:40.571652 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:40.571652 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:40:40.571652 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:40.571652 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 279/550] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 17 16:40:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:40.571652 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035bb60e-b5d0-447a-94a7-49fb7fd3304b None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:40.574296 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035bb60e-b5d0-447a-94a7-49fb7fd3304b None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:40.574296 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035bb60e-b5d0-447a-94a7-49fb7fd3304b None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:40.578425 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 280/551] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:40.581555 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e2652-e180-41aa-9d21-526d6ef971cd None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:40.581555 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e2652-e180-41aa-9d21-526d6ef971cd None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:40.581555 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e2652-e180-41aa-9d21-526d6ef971cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:40.582584 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2b8b0-a833-4c0b-8ddc-24724b3ff6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:40.582584 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2b8b0-a833-4c0b-8ddc-24724b3ff6fc None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:40.582584 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2b8b0-a833-4c0b-8ddc-24724b3ff6fc None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:40.583305 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2b8b0-a833-4c0b-8ddc-24724b3ff6fc None admin] PATH_INFO: `/v3/users/swift` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:40.600654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-d09e2652-e180-41aa-9d21-526d6ef971cd None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:40.601928 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-d09e2652-e180-41aa-9d21-526d6ef971cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:40.607079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e2b8b0-a833-4c0b-8ddc-24724b3ff6fc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:40.610811 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e2b8b0-a833-4c0b-8ddc-24724b3ff6fc None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:40.650755 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-88e2b8b0-a833-4c0b-8ddc-24724b3ff6fc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 17 16:40:40.650755 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:40.650755 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:40.650755 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:40.650755 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:40.650755 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:40.650755 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:40.650755 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:40.650755 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:40.650755 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:40.650755 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:40.650755 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:40.650755 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:40:40.650755 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:40:40.650755 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:40:40.650755 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:40:40.653990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:40.653990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:40.653990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:40:40.653990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:40:40.653990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:40:40.653990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:40:40.653990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:40.653990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:40.653990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:40.653990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:40.653990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:40.653990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:40.653990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:40.653990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:40.653990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:40.653990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:40.653990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:40.653990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:40.656045 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:40.656045 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:40.656045 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:40:40.656045 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:40:40.656045 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:40:40.656045 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:40:40.656045 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:40:40.656045 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:40:40.656045 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 17 16:40:40.656045 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:40.657777 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:40.668826 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 272/552] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:40:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 419 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:40.668826 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2b8b0-a833-4c0b-8ddc-24724b3ff6fc None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:40.709132 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 281/553] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 17 16:40:40.710383 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e2652-e180-41aa-9d21-526d6ef971cd None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:40.841116 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e2652-e180-41aa-9d21-526d6ef971cd None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:40.869982 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:40.879014 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2b8b0-a833-4c0b-8ddc-24724b3ff6fc None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:40.882293 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5cc96-c138-47cb-a64b-89d80b6cd410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:40.883157 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5cc96-c138-47cb-a64b-89d80b6cd410 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:40.889151 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5cc96-c138-47cb-a64b-89d80b6cd410 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:40.889151 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5cc96-c138-47cb-a64b-89d80b6cd410 None admin] PATH_INFO: `/v3/users` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:40.889151 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b5cc96-c138-47cb-a64b-89d80b6cd410 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:40.889151 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b5cc96-c138-47cb-a64b-89d80b6cd410 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:40.889151 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b5cc96-c138-47cb-a64b-89d80b6cd410 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:40.908681 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 282/554] 10.222.0.40 () {56 vars in 1135 bytes} [Sun Jul 17 16:40:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:40.909713 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5cc96-c138-47cb-a64b-89d80b6cd410 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:40.946653 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:40.954977 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3f5d72-9fd5-4290-8a7a-be48684f22d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:40.959513 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f5d72-9fd5-4290-8a7a-be48684f22d8 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:40.959513 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f5d72-9fd5-4290-8a7a-be48684f22d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:40.959513 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3f5d72-9fd5-4290-8a7a-be48684f22d8 None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:40.959513 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3f5d72-9fd5-4290-8a7a-be48684f22d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:40.960595 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3f5d72-9fd5-4290-8a7a-be48684f22d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:40.963468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3f5d72-9fd5-4290-8a7a-be48684f22d8 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:40.982436 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 273/555] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 17 16:40:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:40.984301 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3f5d72-9fd5-4290-8a7a-be48684f22d8 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:41.032902 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5cc96-c138-47cb-a64b-89d80b6cd410 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:41.071005 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5fc10a-3422-4485-8879-7a1d7f5360b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:41.071829 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5fc10a-3422-4485-8879-7a1d7f5360b5 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:41.072122 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5fc10a-3422-4485-8879-7a1d7f5360b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:41.072404 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5fc10a-3422-4485-8879-7a1d7f5360b5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:41.085220 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5fc10a-3422-4485-8879-7a1d7f5360b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:41.093510 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-2d5fc10a-3422-4485-8879-7a1d7f5360b5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:40:41.093510 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:41.093510 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:41.093510 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:41.093510 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:41.093510 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:41.093510 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:41.093510 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:41.093510 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:41.093510 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:41.093510 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:41.093510 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:41.093510 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:41.093510 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:41.093510 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:41.093510 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:41.095056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:41.095056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:41.095056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:41.095056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:41.095056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:41.095056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:41.095056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:41.095056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:41.095056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:41.095056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:41.095056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:41.095056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:41.095056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:41.095056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:41.095056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:41.095056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:41.096247 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:41.096247 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:41.096247 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:41.096247 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:41.096247 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:41.096247 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:41.096247 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:41.096247 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:41.096247 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:41.096247 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:41.096247 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:40:41.096247 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:41.096247 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 283/556] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 17 16:40:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:41.096247 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5fc10a-3422-4485-8879-7a1d7f5360b5 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:41.125544 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3f5d72-9fd5-4290-8a7a-be48684f22d8 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:41.165800 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf08c5-5da9-4681-978e-0869052cb07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:41.167193 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf08c5-5da9-4681-978e-0869052cb07d None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:41.167624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf08c5-5da9-4681-978e-0869052cb07d None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:41.168100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cf08c5-5da9-4681-978e-0869052cb07d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:41.204305 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:41.212288 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:41.223147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5fc10a-3422-4485-8879-7a1d7f5360b5 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:41.253248 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cf08c5-5da9-4681-978e-0869052cb07d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:41.254802 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cf08c5-5da9-4681-978e-0869052cb07d None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:41.262275 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0917fd4-77ce-4dc2-985c-995f3bb15d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:41.263357 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0917fd4-77ce-4dc2-985c-995f3bb15d6e None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:41.263806 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0917fd4-77ce-4dc2-985c-995f3bb15d6e None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:41.264232 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0917fd4-77ce-4dc2-985c-995f3bb15d6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:41.277232 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0917fd4-77ce-4dc2-985c-995f3bb15d6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:41.279070 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0917fd4-77ce-4dc2-985c-995f3bb15d6e None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:41.299271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-a0917fd4-77ce-4dc2-985c-995f3bb15d6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:41.299271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:41.299271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:41.299271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:41.299271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:41.299271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:41.299271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:41.299271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:41.299271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:41.299271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:41.299271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:41.299271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:41.299271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:41.299271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:41.299271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:41.300897 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:41.300897 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:41.300897 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:41.300897 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:41.300897 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:41.300897 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:41.300897 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:41.300897 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:41.300897 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:41.300897 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:41.300897 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:41.300897 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:41.300897 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:41.300897 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:41.300897 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:41.300897 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:41.300897 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:41.302018 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:41.302018 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:41.302018 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:41.302018 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:41.303958 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 284/557] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:41.305105 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0917fd4-77ce-4dc2-985c-995f3bb15d6e None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:41.368373 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 274/558] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:40:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 17 16:40:41.369507 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf08c5-5da9-4681-978e-0869052cb07d None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:41.417326 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0917fd4-77ce-4dc2-985c-995f3bb15d6e None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:41.434839 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:41.457206 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f4152-769f-484c-a710-49c0149726b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:41.458296 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f4152-769f-484c-a710-49c0149726b6 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:41.458800 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f4152-769f-484c-a710-49c0149726b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:41.459294 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f4152-769f-484c-a710-49c0149726b6 None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:41.460282 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f4152-769f-484c-a710-49c0149726b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:41.460837 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f4152-769f-484c-a710-49c0149726b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:41.462481 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f4152-769f-484c-a710-49c0149726b6 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:41.477772 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 285/559] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 17 16:40:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:41.479795 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f4152-769f-484c-a710-49c0149726b6 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:41.482395 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cf08c5-5da9-4681-978e-0869052cb07d None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:41.490578 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:41.520510 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74591b42-c7ce-4481-a043-1693eef0f077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:41.521219 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74591b42-c7ce-4481-a043-1693eef0f077 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:41.521507 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74591b42-c7ce-4481-a043-1693eef0f077 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:41.521789 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74591b42-c7ce-4481-a043-1693eef0f077 None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:41.522531 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74591b42-c7ce-4481-a043-1693eef0f077 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:41.522842 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74591b42-c7ce-4481-a043-1693eef0f077 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:41.524727 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74591b42-c7ce-4481-a043-1693eef0f077 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:41.538298 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 275/560] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:41.539599 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74591b42-c7ce-4481-a043-1693eef0f077 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:41.603689 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f4152-769f-484c-a710-49c0149726b6 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:41.643655 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08760ae-9747-4af7-8fed-e9562d5070fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:41.645024 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08760ae-9747-4af7-8fed-e9562d5070fd None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:41.645539 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08760ae-9747-4af7-8fed-e9562d5070fd None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:41.646026 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08760ae-9747-4af7-8fed-e9562d5070fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:41.651292 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74591b42-c7ce-4481-a043-1693eef0f077 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:41.659966 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08760ae-9747-4af7-8fed-e9562d5070fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:41.661912 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08760ae-9747-4af7-8fed-e9562d5070fd None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:41.676100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:41.684579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-f08760ae-9747-4af7-8fed-e9562d5070fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:41.684579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:41.684579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:41.684579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:41.684579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:41.684579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:41.684579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:41.684579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:41.684579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:41.684579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:41.684579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:41.684579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:41.684579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:41.684579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:41.684579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:41.690326 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:41.690326 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:41.690326 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:41.690326 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:41.690326 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:41.690326 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:41.690326 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:41.690326 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:41.690326 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:41.690326 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:41.690326 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:41.690326 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:41.690326 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:41.690326 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:41.690326 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:41.690326 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:41.690326 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:41.691228 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:41.691228 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:41.691228 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:41.691228 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:41.691228 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 286/561] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:41.691228 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08760ae-9747-4af7-8fed-e9562d5070fd None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:41.696342 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703d28b-44db-4605-a808-10ce9e92c1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:41.697500 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703d28b-44db-4605-a808-10ce9e92c1e4 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:41.697985 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703d28b-44db-4605-a808-10ce9e92c1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:41.698498 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703d28b-44db-4605-a808-10ce9e92c1e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:41.711541 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d703d28b-44db-4605-a808-10ce9e92c1e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:41.713349 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d703d28b-44db-4605-a808-10ce9e92c1e4 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:41.733723 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-d703d28b-44db-4605-a808-10ce9e92c1e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:41.733723 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:41.733723 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:41.733723 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:41.733723 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:41.733723 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:41.733723 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:41.733723 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:41.733723 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:41.733723 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:41.733723 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:41.733723 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:41.733723 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:41.733723 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:41.733723 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:41.735345 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:41.735345 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:41.735345 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:41.735345 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:41.735345 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:41.735345 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:41.735345 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:41.735345 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:41.735345 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:41.735345 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:41.735345 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:41.735345 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:41.735345 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:41.735345 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:41.735345 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:41.735345 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:41.735345 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:41.736345 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:41.736345 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:41.736345 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:41.736345 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:41.742974 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 276/562] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:41.744180 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703d28b-44db-4605-a808-10ce9e92c1e4 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:41.761252 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:41.811650 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08760ae-9747-4af7-8fed-e9562d5070fd None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:41.852167 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07356fd0-eec1-4cc3-a8d9-492f31dcdbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:41.853179 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07356fd0-eec1-4cc3-a8d9-492f31dcdbc9 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:41.853569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07356fd0-eec1-4cc3-a8d9-492f31dcdbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:41.853951 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07356fd0-eec1-4cc3-a8d9-492f31dcdbc9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:41.858678 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703d28b-44db-4605-a808-10ce9e92c1e4 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:41.863728 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07356fd0-eec1-4cc3-a8d9-492f31dcdbc9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:41.865408 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07356fd0-eec1-4cc3-a8d9-492f31dcdbc9 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:41.882124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-07356fd0-eec1-4cc3-a8d9-492f31dcdbc9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 17 16:40:41.882124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:41.882124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:41.882124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:41.882124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:41.882124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:41.882124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:41.882124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:41.882124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:41.882124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:41.882124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:41.882124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:41.882124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:40:41.882124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:40:41.882124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:40:41.882124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:40:41.883306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:41.883306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:41.883306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:40:41.883306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:40:41.883306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:40:41.883306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:40:41.883306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:41.883306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:41.883306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:41.883306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:41.883306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:41.883306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:41.883306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:41.883306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:41.883306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:41.883306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:41.883306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:41.883306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:41.884218 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:41.884218 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:41.884218 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:40:41.884218 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:40:41.884218 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:40:41.884218 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:40:41.884218 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:40:41.884218 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:40:41.884218 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 17 16:40:41.884218 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:41.884757 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 287/563] 10.222.0.40 () {56 vars in 1093 bytes} [Sun Jul 17 16:40:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:41.885877 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07356fd0-eec1-4cc3-a8d9-492f31dcdbc9 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:41.901487 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd8814-1806-4900-bda7-73f0bf4ce4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:41.902543 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd8814-1806-4900-bda7-73f0bf4ce4d2 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:41.902978 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd8814-1806-4900-bda7-73f0bf4ce4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:41.903420 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd8814-1806-4900-bda7-73f0bf4ce4d2 None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:41.904351 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd8814-1806-4900-bda7-73f0bf4ce4d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:41.904833 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd8814-1806-4900-bda7-73f0bf4ce4d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:41.906471 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd8814-1806-4900-bda7-73f0bf4ce4d2 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:41.921635 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 277/564] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:41.922906 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd8814-1806-4900-bda7-73f0bf4ce4d2 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:41.958627 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:42.005819 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07356fd0-eec1-4cc3-a8d9-492f31dcdbc9 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:42.026469 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:42.038028 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd8814-1806-4900-bda7-73f0bf4ce4d2 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:42.046425 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae5af27-5d1a-49d8-b379-1f8b1497b8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:42.047670 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae5af27-5d1a-49d8-b379-1f8b1497b8b7 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:42.047841 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae5af27-5d1a-49d8-b379-1f8b1497b8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:42.048217 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae5af27-5d1a-49d8-b379-1f8b1497b8b7 None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:42.049108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae5af27-5d1a-49d8-b379-1f8b1497b8b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:42.049557 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae5af27-5d1a-49d8-b379-1f8b1497b8b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:42.051186 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae5af27-5d1a-49d8-b379-1f8b1497b8b7 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:42.072061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 288/565] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:42.072061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae5af27-5d1a-49d8-b379-1f8b1497b8b7 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:42.084926 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079241d5-c6cd-454d-8d56-eb5296ccda79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:42.085945 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079241d5-c6cd-454d-8d56-eb5296ccda79 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:42.086437 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079241d5-c6cd-454d-8d56-eb5296ccda79 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:42.086906 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079241d5-c6cd-454d-8d56-eb5296ccda79 None admin] PATH_INFO: `/v3/users` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:42.087926 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079241d5-c6cd-454d-8d56-eb5296ccda79 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:42.088400 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079241d5-c6cd-454d-8d56-eb5296ccda79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:42.090153 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079241d5-c6cd-454d-8d56-eb5296ccda79 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:42.110830 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 278/566] 10.222.0.40 () {56 vars in 1133 bytes} [Sun Jul 17 16:40:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:42.111999 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079241d5-c6cd-454d-8d56-eb5296ccda79 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:42.187598 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae5af27-5d1a-49d8-b379-1f8b1497b8b7 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:42.221313 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:42.229980 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e1a51-4217-4dfe-b8ba-8fad19abde43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:42.232960 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e1a51-4217-4dfe-b8ba-8fad19abde43 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:42.233087 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079241d5-c6cd-454d-8d56-eb5296ccda79 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:42.233170 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e1a51-4217-4dfe-b8ba-8fad19abde43 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:42.233170 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e1a51-4217-4dfe-b8ba-8fad19abde43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:42.240873 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e1a51-4217-4dfe-b8ba-8fad19abde43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:42.242735 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e1a51-4217-4dfe-b8ba-8fad19abde43 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:42.259472 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-cb3e1a51-4217-4dfe-b8ba-8fad19abde43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:40:42.259472 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:42.259472 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:42.259472 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:42.259472 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:42.259472 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:42.259472 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:42.259472 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:42.259472 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:42.259472 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:42.259472 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:42.259472 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:42.259472 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:40:42.259472 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:40:42.259472 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:40:42.260441 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:40:42.260441 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:42.260441 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:42.260441 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:42.260441 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:42.260441 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:42.260441 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:42.260441 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:42.260441 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:42.260441 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:42.260441 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:42.260441 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:42.260441 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:42.260441 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:42.260441 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:42.260441 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:42.260441 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:42.261401 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:40:42.261401 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:40:42.261401 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:40:42.261401 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:40:42.261401 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:40:42.261401 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:40:42.261401 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:40:42.261401 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:42.263676 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 289/567] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:40:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:42.264815 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e1a51-4217-4dfe-b8ba-8fad19abde43 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:42.270981 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c29a0-d9fe-4077-8089-025b4d13ed54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:42.271216 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c29a0-d9fe-4077-8089-025b4d13ed54 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:42.271446 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c29a0-d9fe-4077-8089-025b4d13ed54 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:42.271728 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c29a0-d9fe-4077-8089-025b4d13ed54 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:42.280160 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7c29a0-d9fe-4077-8089-025b4d13ed54 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:42.282264 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7c29a0-d9fe-4077-8089-025b4d13ed54 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:42.298626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-1e7c29a0-d9fe-4077-8089-025b4d13ed54 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 17 16:40:42.298626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:42.298626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:42.298626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:42.298626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:42.298626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:42.298626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:42.298626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:42.298626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:42.298626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:42.298626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:42.298626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:42.298626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:40:42.298626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:40:42.298626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:40:42.298626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:40:42.299822 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:42.299822 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:42.299822 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:40:42.299822 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:40:42.299822 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:40:42.299822 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:40:42.299822 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:42.299822 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:42.299822 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:42.299822 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:42.299822 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:42.299822 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:42.299822 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:42.299822 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:42.299822 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:42.299822 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:42.299822 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:42.299822 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:42.300931 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:42.300931 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:42.300931 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:40:42.300931 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:40:42.300931 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:40:42.300931 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:40:42.300931 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:40:42.300931 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:40:42.300931 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 17 16:40:42.300931 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:42.304703 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 279/568] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:40:42 2022] GET /identity/v3/users/glance => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:42.305784 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c29a0-d9fe-4077-8089-025b4d13ed54 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:42.333765 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:42.413998 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e1a51-4217-4dfe-b8ba-8fad19abde43 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:42.443401 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c29a0-d9fe-4077-8089-025b4d13ed54 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:42.474328 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9562bf30-e51c-4e25-a916-9bccace1d09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:42.475257 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562bf30-e51c-4e25-a916-9bccace1d09f None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:42.475604 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562bf30-e51c-4e25-a916-9bccace1d09f None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:42.475994 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562bf30-e51c-4e25-a916-9bccace1d09f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:42.487537 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f412d6e-94c3-416f-8ddc-33073439e7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:42.488445 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f412d6e-94c3-416f-8ddc-33073439e7e7 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:42.488940 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f412d6e-94c3-416f-8ddc-33073439e7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:42.489218 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f412d6e-94c3-416f-8ddc-33073439e7e7 None admin] PATH_INFO: `/v3/projects` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:42.490027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f412d6e-94c3-416f-8ddc-33073439e7e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:42.490336 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f412d6e-94c3-416f-8ddc-33073439e7e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:42.491793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f412d6e-94c3-416f-8ddc-33073439e7e7 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:42.496591 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9562bf30-e51c-4e25-a916-9bccace1d09f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:42.499470 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9562bf30-e51c-4e25-a916-9bccace1d09f None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:42.511742 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 280/569] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:40:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:42.512943 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f412d6e-94c3-416f-8ddc-33073439e7e7 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:42.520899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-9562bf30-e51c-4e25-a916-9bccace1d09f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:42.520899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:42.520899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:42.520899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:42.520899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:42.520899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:42.520899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:42.520899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:42.520899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:42.520899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:42.520899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:42.520899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:42.520899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:42.520899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:42.520899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:42.522626 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:42.522626 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:42.522626 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:42.522626 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:42.522626 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:42.522626 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:42.522626 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:42.522626 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:42.522626 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:42.522626 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:42.522626 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:42.522626 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:42.522626 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:42.522626 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:42.522626 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:42.522626 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:42.522626 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:42.523992 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:42.523992 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:42.523992 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:42.523992 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:42.523992 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 290/570] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:42.523992 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d50938-faf4-4022-9d58-59ef3dfccb02 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:42.523992 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d50938-faf4-4022-9d58-59ef3dfccb02 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:42.523992 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d50938-faf4-4022-9d58-59ef3dfccb02 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:42.529091 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 291/571] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:42.530286 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d50938-faf4-4022-9d58-59ef3dfccb02 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:42.535681 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:42.659266 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f412d6e-94c3-416f-8ddc-33073439e7e7 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:42.681024 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:42.701447 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d50938-faf4-4022-9d58-59ef3dfccb02 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:42.710465 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79b849-93ab-4806-9424-df48e7c12b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:42.711161 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79b849-93ab-4806-9424-df48e7c12b1a None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:42.711446 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79b849-93ab-4806-9424-df48e7c12b1a None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:42.711722 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79b849-93ab-4806-9424-df48e7c12b1a None admin] PATH_INFO: `/v3/users` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:42.712690 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e79b849-93ab-4806-9424-df48e7c12b1a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:42.713036 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e79b849-93ab-4806-9424-df48e7c12b1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:42.714553 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e79b849-93ab-4806-9424-df48e7c12b1a None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:42.737433 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 281/572] 10.222.0.40 () {56 vars in 1137 bytes} [Sun Jul 17 16:40:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:42.738336 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79b849-93ab-4806-9424-df48e7c12b1a None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:42.749851 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34999432-358f-4fda-a19c-12a57fcb403d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:42.751307 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34999432-358f-4fda-a19c-12a57fcb403d None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:42.751598 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34999432-358f-4fda-a19c-12a57fcb403d None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:42.751875 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34999432-358f-4fda-a19c-12a57fcb403d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:42.752731 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34999432-358f-4fda-a19c-12a57fcb403d None admin] RBAC: Adding query filter params (role.id=adb9c27c6e5246fdb0cf386358a822cd, scope.project.id=927057f9a5af4833aeac9992203a223c, user.id=69482eeb1e8547aabca9f036b5c34f13) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:42.753074 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34999432-358f-4fda-a19c-12a57fcb403d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:42.754548 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34999432-358f-4fda-a19c-12a57fcb403d None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:42.760832 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 292/573] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:40:42 2022] GET /identity/v3/role_assignments?user.id=69482eeb1e8547aabca9f036b5c34f13&scope.project.id=927057f9a5af4833aeac9992203a223c&role.id=adb9c27c6e5246fdb0cf386358a822cd => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:42.762158 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060b95b-24c3-4778-985a-8555a585c6c4 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:42.762449 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060b95b-24c3-4778-985a-8555a585c6c4 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:42.762730 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060b95b-24c3-4778-985a-8555a585c6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:42.788259 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-d060b95b-24c3-4778-985a-8555a585c6c4 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:42.789619 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-d060b95b-24c3-4778-985a-8555a585c6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:42.841422 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:42.901010 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 293/574] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:42.902630 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b584e7a-77e2-41dc-8d68-0c1c6a8f2ee9 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:42.902995 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b584e7a-77e2-41dc-8d68-0c1c6a8f2ee9 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:42.903283 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b584e7a-77e2-41dc-8d68-0c1c6a8f2ee9 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:42.903721 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79b849-93ab-4806-9424-df48e7c12b1a None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:42.908859 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 294/575] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:42.908859 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b584e7a-77e2-41dc-8d68-0c1c6a8f2ee9 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:42.965785 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd7d47-4679-4c70-b406-7c4f54b1abf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:42.966466 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd7d47-4679-4c70-b406-7c4f54b1abf1 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:42.966754 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd7d47-4679-4c70-b406-7c4f54b1abf1 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:42.967039 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd7d47-4679-4c70-b406-7c4f54b1abf1 None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:42.967784 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd7d47-4679-4c70-b406-7c4f54b1abf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:42.968096 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd7d47-4679-4c70-b406-7c4f54b1abf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:42.970555 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd7d47-4679-4c70-b406-7c4f54b1abf1 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:42.991880 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 282/576] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:42.993085 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6374f2-c956-42e5-818d-34035a4f826c None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:42.993383 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6374f2-c956-42e5-818d-34035a4f826c None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:42.993665 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6374f2-c956-42e5-818d-34035a4f826c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:43.012368 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-0c6374f2-c956-42e5-818d-34035a4f826c None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:43.013658 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-0c6374f2-c956-42e5-818d-34035a4f826c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:43.045095 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:43.065997 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b584e7a-77e2-41dc-8d68-0c1c6a8f2ee9 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:43.105235 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 283/577] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:43.105235 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6374f2-c956-42e5-818d-34035a4f826c None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:43.110648 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204daece-f1bd-467d-a251-0e884a0f00cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:43.111574 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204daece-f1bd-467d-a251-0e884a0f00cb None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:43.111865 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204daece-f1bd-467d-a251-0e884a0f00cb None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:43.112152 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204daece-f1bd-467d-a251-0e884a0f00cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:43.136774 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204daece-f1bd-467d-a251-0e884a0f00cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:43.138328 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204daece-f1bd-467d-a251-0e884a0f00cb None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:43.167696 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-204daece-f1bd-467d-a251-0e884a0f00cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:43.167696 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:43.167696 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:43.167696 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:43.167696 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:43.167696 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:43.167696 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:43.167696 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:43.167696 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:43.167696 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:43.167696 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:43.167696 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:43.167696 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:43.167696 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:43.167696 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:43.169422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:43.169422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:43.169422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:43.169422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:43.169422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:43.169422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:43.169422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:43.169422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:43.169422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:43.169422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:43.169422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:43.169422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:43.169422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:43.169422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:43.169422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:43.169422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:43.169422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:43.171223 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:43.171223 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:43.171223 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:43.171223 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:43.174659 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 295/578] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:43.176256 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204daece-f1bd-467d-a251-0e884a0f00cb None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:43.222721 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:43.290651 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6374f2-c956-42e5-818d-34035a4f826c None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:43.316516 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204daece-f1bd-467d-a251-0e884a0f00cb None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:43.347367 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a6c5cc-aea4-4987-8555-4fe91525bca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:43.348980 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6c5cc-aea4-4987-8555-4fe91525bca1 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:43.349568 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6c5cc-aea4-4987-8555-4fe91525bca1 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:43.350094 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6c5cc-aea4-4987-8555-4fe91525bca1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:43.367916 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a6c5cc-aea4-4987-8555-4fe91525bca1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:43.376571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-78a6c5cc-aea4-4987-8555-4fe91525bca1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:40:43.376571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:43.376571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:43.376571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:43.376571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:43.376571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:43.376571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:43.376571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:43.376571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:43.376571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:43.376571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:43.376571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:43.376571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:43.376571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:43.376571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:43.376571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:43.377990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:43.377990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:43.377990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:43.377990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:43.377990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:43.377990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:43.377990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:43.377990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:43.377990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:43.377990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:43.377990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:43.377990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:43.377990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:43.377990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:43.377990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:43.377990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:43.379133 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:43.379133 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:43.379133 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:43.379133 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:43.379133 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:43.379133 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:43.379133 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:43.379133 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:43.379133 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:43.379133 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:43.379133 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:40:43.379133 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:43.379133 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 284/579] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 17 16:40:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:43.380550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a6c5cc-aea4-4987-8555-4fe91525bca1 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:43.382184 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96df040-2283-490b-8aab-2531e69365e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:43.383722 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96df040-2283-490b-8aab-2531e69365e5 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:43.384368 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96df040-2283-490b-8aab-2531e69365e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:43.385010 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96df040-2283-490b-8aab-2531e69365e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:43.389890 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:43.396336 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96df040-2283-490b-8aab-2531e69365e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:43.398071 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96df040-2283-490b-8aab-2531e69365e5 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:43.414434 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-b96df040-2283-490b-8aab-2531e69365e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:40:43.414434 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:43.414434 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:43.414434 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:43.414434 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:43.414434 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:43.414434 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:43.414434 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:43.414434 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:43.414434 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:43.414434 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:43.414434 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:43.414434 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:40:43.414434 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:40:43.414434 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:40:43.416569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:40:43.416569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:43.416569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:43.416569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:43.416569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:43.416569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:43.416569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:43.416569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:43.416569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:43.416569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:43.416569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:43.416569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:43.416569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:43.416569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:43.416569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:43.416569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:43.416569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:43.418519 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:40:43.418519 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:40:43.418519 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:40:43.418519 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:40:43.418519 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:40:43.418519 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:40:43.418519 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:40:43.418519 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:43.418519 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 296/580] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:40:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:43.418519 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96df040-2283-490b-8aab-2531e69365e5 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:43.565798 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96df040-2283-490b-8aab-2531e69365e5 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:43.586790 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a6c5cc-aea4-4987-8555-4fe91525bca1 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:43.610612 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a24a1-609c-460d-9e41-422bafe5248b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:43.611293 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a24a1-609c-460d-9e41-422bafe5248b None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:43.611595 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a24a1-609c-460d-9e41-422bafe5248b None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:43.611870 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a24a1-609c-460d-9e41-422bafe5248b None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:43.612625 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a24a1-609c-460d-9e41-422bafe5248b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:43.612954 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a24a1-609c-460d-9e41-422bafe5248b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:43.614382 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a24a1-609c-460d-9e41-422bafe5248b None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:43.620734 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:43.635104 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 297/581] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:43.635104 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a24a1-609c-460d-9e41-422bafe5248b None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:43.657961 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4eabb3-570b-40e7-b492-cd6bf26204da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:43.659343 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4eabb3-570b-40e7-b492-cd6bf26204da None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:43.659873 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4eabb3-570b-40e7-b492-cd6bf26204da None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:43.660383 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4eabb3-570b-40e7-b492-cd6bf26204da None admin] PATH_INFO: `/v3/roles/service` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:43.678876 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4eabb3-570b-40e7-b492-cd6bf26204da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:43.692433 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-3f4eabb3-570b-40e7-b492-cd6bf26204da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:40:43.692433 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:43.692433 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:43.692433 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:43.692433 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:43.692433 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:43.692433 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:43.692433 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:43.692433 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:43.692433 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:43.692433 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:43.692433 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:43.692433 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:43.692433 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:43.692433 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:43.692433 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:43.703165 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:43.703165 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:43.703165 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:43.703165 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:43.703165 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:43.703165 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:43.703165 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:43.703165 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:43.703165 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:43.703165 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:43.703165 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:43.703165 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:43.703165 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:43.703165 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:43.703165 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:43.703165 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:43.705729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:43.705729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:43.705729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:43.705729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:43.705729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:43.705729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:43.705729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:43.705729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:43.705729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:43.705729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:43.705729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:40:43.705729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:43.705729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 285/582] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 17 16:40:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:43.705729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4eabb3-570b-40e7-b492-cd6bf26204da None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:43.754112 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:43.819625 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a24a1-609c-460d-9e41-422bafe5248b None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:43.861682 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc454813-74bc-4c49-8d99-79a8762b4db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:43.862601 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc454813-74bc-4c49-8d99-79a8762b4db0 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:43.863077 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc454813-74bc-4c49-8d99-79a8762b4db0 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:43.863446 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc454813-74bc-4c49-8d99-79a8762b4db0 None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:43.864354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc454813-74bc-4c49-8d99-79a8762b4db0 None admin] RBAC: Adding query filter params (name=service) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:43.864843 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc454813-74bc-4c49-8d99-79a8762b4db0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:43.866682 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc454813-74bc-4c49-8d99-79a8762b4db0 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:43.884155 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 298/583] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 17 16:40:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:43.884155 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc454813-74bc-4c49-8d99-79a8762b4db0 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:43.907299 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4eabb3-570b-40e7-b492-cd6bf26204da None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:43.949071 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02214048-c881-48c7-86e9-9e259f3f4d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:43.949790 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02214048-c881-48c7-86e9-9e259f3f4d6c None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:43.950030 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02214048-c881-48c7-86e9-9e259f3f4d6c None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:43.950316 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02214048-c881-48c7-86e9-9e259f3f4d6c None admin] PATH_INFO: `/v3/projects` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:43.951136 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02214048-c881-48c7-86e9-9e259f3f4d6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:43.951456 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02214048-c881-48c7-86e9-9e259f3f4d6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:43.952966 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02214048-c881-48c7-86e9-9e259f3f4d6c None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:43.967881 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 286/584] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:40:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:43.968843 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02214048-c881-48c7-86e9-9e259f3f4d6c None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:44.020342 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:44.037825 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc454813-74bc-4c49-8d99-79a8762b4db0 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:44.069451 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:44.099339 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1aea1e-e69c-47d9-9770-ed950c74a539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:44.100505 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1aea1e-e69c-47d9-9770-ed950c74a539 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:44.100920 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1aea1e-e69c-47d9-9770-ed950c74a539 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:44.101318 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1aea1e-e69c-47d9-9770-ed950c74a539 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:44.114329 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1aea1e-e69c-47d9-9770-ed950c74a539 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:44.116669 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1aea1e-e69c-47d9-9770-ed950c74a539 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:44.123751 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02214048-c881-48c7-86e9-9e259f3f4d6c None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:44.140870 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-fa1aea1e-e69c-47d9-9770-ed950c74a539 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:40:44.140870 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:44.140870 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:44.140870 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:44.140870 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:44.140870 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:44.140870 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:44.140870 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:44.140870 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:44.140870 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:44.140870 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:44.140870 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:44.140870 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:40:44.140870 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:40:44.140870 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:40:44.143085 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:40:44.143085 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:44.143085 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:44.143085 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:44.143085 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:44.143085 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:44.143085 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:44.143085 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:44.143085 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:44.143085 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:44.143085 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:44.143085 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:44.143085 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:44.143085 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:44.143085 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:44.143085 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:44.143085 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:44.147089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:40:44.147089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:40:44.147089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:40:44.147089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:40:44.147089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:40:44.147089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:40:44.147089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:40:44.147089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:44.147089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 299/585] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:40:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:44.147089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1aea1e-e69c-47d9-9770-ed950c74a539 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:44.205072 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc646f3-ad51-44bd-b529-4721f09f4346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:44.205961 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc646f3-ad51-44bd-b529-4721f09f4346 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:44.206299 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc646f3-ad51-44bd-b529-4721f09f4346 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:44.206650 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc646f3-ad51-44bd-b529-4721f09f4346 None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:44.207510 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc646f3-ad51-44bd-b529-4721f09f4346 None admin] RBAC: Adding query filter params (name=service) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:44.207890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc646f3-ad51-44bd-b529-4721f09f4346 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:44.209405 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc646f3-ad51-44bd-b529-4721f09f4346 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:44.220572 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 287/586] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 17 16:40:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:44.221497 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc646f3-ad51-44bd-b529-4721f09f4346 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:44.343554 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:44.347263 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc646f3-ad51-44bd-b529-4721f09f4346 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:44.357771 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1aea1e-e69c-47d9-9770-ed950c74a539 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:44.397660 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1325ca52-f4cf-46a0-b349-a78fd3adf36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:44.399250 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325ca52-f4cf-46a0-b349-a78fd3adf36d None admin] REQUEST_METHOD: `PUT` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:44.399762 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325ca52-f4cf-46a0-b349-a78fd3adf36d None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:44.400246 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325ca52-f4cf-46a0-b349-a78fd3adf36d None admin] PATH_INFO: `/v3/projects/927057f9a5af4833aeac9992203a223c/users/03244087f1a34bddb11ea46aba11a148/roles/adb9c27c6e5246fdb0cf386358a822cd` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:44.412557 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423fd94-ca49-44e0-8e7c-83ee18531b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:44.414019 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423fd94-ca49-44e0-8e7c-83ee18531b21 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:44.414535 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423fd94-ca49-44e0-8e7c-83ee18531b21 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:44.415182 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423fd94-ca49-44e0-8e7c-83ee18531b21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:44.421793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1325ca52-f4cf-46a0-b349-a78fd3adf36d None admin] RBAC: Authorizing `identity:create_grant(project_id=927057f9a5af4833aeac9992203a223c, user_id=03244087f1a34bddb11ea46aba11a148, role_id=adb9c27c6e5246fdb0cf386358a822cd)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:44.423952 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1325ca52-f4cf-46a0-b349-a78fd3adf36d None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:44.435651 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a423fd94-ca49-44e0-8e7c-83ee18531b21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:44.439028 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a423fd94-ca49-44e0-8e7c-83ee18531b21 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:44.475223 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-a423fd94-ca49-44e0-8e7c-83ee18531b21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:44.475223 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:44.475223 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:44.475223 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:44.475223 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:44.475223 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:44.475223 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:44.475223 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:44.475223 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:44.475223 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:44.475223 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:44.475223 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:44.475223 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:44.475223 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:44.475223 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:44.476828 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:44.476828 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:44.476828 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:44.476828 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:44.476828 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:44.476828 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:44.476828 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:44.476828 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:44.476828 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:44.476828 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:44.476828 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:44.476828 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:44.476828 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:44.476828 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:44.476828 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:44.476828 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:44.476828 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:44.478490 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:44.478490 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:44.478490 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:44.478490 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:44.478490 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 300/587] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:44.479453 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423fd94-ca49-44e0-8e7c-83ee18531b21 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:44.513209 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 288/588] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 17 16:40:44 2022] PUT /identity/v3/projects/927057f9a5af4833aeac9992203a223c/users/03244087f1a34bddb11ea46aba11a148/roles/adb9c27c6e5246fdb0cf386358a822cd => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 17 16:40:44.514806 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1325ca52-f4cf-46a0-b349-a78fd3adf36d None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:44.549119 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:44.662796 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423fd94-ca49-44e0-8e7c-83ee18531b21 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:44.752547 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396a285-38a6-4e11-927c-64b746477c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:44.754318 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396a285-38a6-4e11-927c-64b746477c39 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:44.754867 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396a285-38a6-4e11-927c-64b746477c39 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:44.755409 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396a285-38a6-4e11-927c-64b746477c39 None admin] PATH_INFO: `/v3/projects` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:44.756936 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9396a285-38a6-4e11-927c-64b746477c39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:44.757952 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9396a285-38a6-4e11-927c-64b746477c39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:44.761508 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9396a285-38a6-4e11-927c-64b746477c39 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:44.766482 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1325ca52-f4cf-46a0-b349-a78fd3adf36d None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:44.791365 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 301/589] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:40:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:44.793284 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396a285-38a6-4e11-927c-64b746477c39 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:44.844731 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ba89b-1e16-4c83-92a9-17cbc13275fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:44.846199 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ba89b-1e16-4c83-92a9-17cbc13275fd None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:44.846709 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ba89b-1e16-4c83-92a9-17cbc13275fd None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:44.847223 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ba89b-1e16-4c83-92a9-17cbc13275fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:44.876317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985ba89b-1e16-4c83-92a9-17cbc13275fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:44.879752 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985ba89b-1e16-4c83-92a9-17cbc13275fd None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:44.918429 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-985ba89b-1e16-4c83-92a9-17cbc13275fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:44.918429 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:44.918429 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:44.918429 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:44.918429 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:44.918429 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:44.918429 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:44.918429 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:44.918429 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:44.918429 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:44.918429 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:44.918429 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:44.918429 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:44.918429 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:44.918429 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:44.920060 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:44.920060 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:44.920060 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:44.920060 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:44.920060 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:44.920060 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:44.920060 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:44.920060 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:44.920060 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:44.920060 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:44.920060 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:44.920060 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:44.920060 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:44.920060 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:44.920060 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:44.920060 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:44.920060 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:44.921744 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:44.921744 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:44.921744 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:44.921744 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:44.926015 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 289/590] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 412 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:44.927889 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ba89b-1e16-4c83-92a9-17cbc13275fd None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:44.932877 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:45.002671 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:45.016549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396a285-38a6-4e11-927c-64b746477c39 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:45.054986 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f715c4-fd6c-4757-a0d4-aa1c702086b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:45.055672 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f715c4-fd6c-4757-a0d4-aa1c702086b0 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:45.055958 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f715c4-fd6c-4757-a0d4-aa1c702086b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:45.056236 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f715c4-fd6c-4757-a0d4-aa1c702086b0 None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:45.057025 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f715c4-fd6c-4757-a0d4-aa1c702086b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:45.057336 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f715c4-fd6c-4757-a0d4-aa1c702086b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:45.058770 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f715c4-fd6c-4757-a0d4-aa1c702086b0 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:45.072768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 302/591] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:45.073661 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f715c4-fd6c-4757-a0d4-aa1c702086b0 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:45.118821 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ba89b-1e16-4c83-92a9-17cbc13275fd None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:45.179067 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827b55d-3fa5-4cdf-ac43-9296b31864e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:45.179889 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827b55d-3fa5-4cdf-ac43-9296b31864e8 None admin] REQUEST_METHOD: `PUT` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:45.180173 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827b55d-3fa5-4cdf-ac43-9296b31864e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:45.180453 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827b55d-3fa5-4cdf-ac43-9296b31864e8 None admin] PATH_INFO: `/v3/projects/927057f9a5af4833aeac9992203a223c/users/9e13bcc0f657445cbbdea5b78563f0d3/roles/adb9c27c6e5246fdb0cf386358a822cd` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:45.198260 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d827b55d-3fa5-4cdf-ac43-9296b31864e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=927057f9a5af4833aeac9992203a223c, user_id=9e13bcc0f657445cbbdea5b78563f0d3, role_id=adb9c27c6e5246fdb0cf386358a822cd)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:45.199714 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d827b55d-3fa5-4cdf-ac43-9296b31864e8 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:45.204928 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f715c4-fd6c-4757-a0d4-aa1c702086b0 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:45.242228 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fcb712-db93-4175-9686-315c183019f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:45.243026 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fcb712-db93-4175-9686-315c183019f7 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:45.243360 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fcb712-db93-4175-9686-315c183019f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:45.243677 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fcb712-db93-4175-9686-315c183019f7 None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:45.244453 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fcb712-db93-4175-9686-315c183019f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:45.244838 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fcb712-db93-4175-9686-315c183019f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:45.246366 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fcb712-db93-4175-9686-315c183019f7 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:45.260976 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 303/592] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:45.260976 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5b87b-1288-491f-988b-5af1d3bf1b59 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:45.260976 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5b87b-1288-491f-988b-5af1d3bf1b59 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:45.260976 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5b87b-1288-491f-988b-5af1d3bf1b59 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:45.260976 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 304/593] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:45.260976 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5b87b-1288-491f-988b-5af1d3bf1b59 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:45.288844 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 290/594] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 17 16:40:44 2022] PUT /identity/v3/projects/927057f9a5af4833aeac9992203a223c/users/9e13bcc0f657445cbbdea5b78563f0d3/roles/adb9c27c6e5246fdb0cf386358a822cd => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 17 16:40:45.288844 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827b55d-3fa5-4cdf-ac43-9296b31864e8 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:45.297800 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:45.393352 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:45.490357 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5b87b-1288-491f-988b-5af1d3bf1b59 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:45.507924 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827b55d-3fa5-4cdf-ac43-9296b31864e8 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:45.540528 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de1a5ab-6c58-4bce-921f-038aed3df2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:45.541470 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de1a5ab-6c58-4bce-921f-038aed3df2f9 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:45.541835 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de1a5ab-6c58-4bce-921f-038aed3df2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:45.542174 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de1a5ab-6c58-4bce-921f-038aed3df2f9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:45.552634 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de1a5ab-6c58-4bce-921f-038aed3df2f9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:45.554573 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de1a5ab-6c58-4bce-921f-038aed3df2f9 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:45.576148 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-1de1a5ab-6c58-4bce-921f-038aed3df2f9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 17 16:40:45.576148 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:45.576148 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:45.576148 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:45.576148 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:45.576148 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:45.576148 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:45.576148 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:45.576148 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:45.576148 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:45.576148 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:45.576148 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:45.576148 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:40:45.576148 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:40:45.576148 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:40:45.576148 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:40:45.577894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:45.577894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:45.577894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:40:45.577894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:40:45.577894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:40:45.577894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:40:45.577894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:45.577894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:45.577894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:45.577894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:45.577894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:45.577894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:45.577894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:45.577894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:45.577894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:45.577894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:45.577894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:45.577894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:45.580270 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:45.580270 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:45.580270 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:40:45.580270 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:40:45.580270 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:40:45.580270 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:40:45.580270 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:40:45.580270 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:40:45.580270 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 17 16:40:45.580270 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:45.580270 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 305/595] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 17 16:40:45 2022] GET /identity/v3/users/neutron => generated 85 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:45.580270 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053f4ce-0e76-4421-9c44-b861405b281c None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:45.580270 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053f4ce-0e76-4421-9c44-b861405b281c None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:45.581647 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053f4ce-0e76-4421-9c44-b861405b281c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:45.585945 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33394163-5c75-4b88-ab6a-f94d8da53814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:45.587775 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33394163-5c75-4b88-ab6a-f94d8da53814 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:45.588676 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33394163-5c75-4b88-ab6a-f94d8da53814 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:45.589463 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33394163-5c75-4b88-ab6a-f94d8da53814 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:45.605021 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-8053f4ce-0e76-4421-9c44-b861405b281c None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:45.606280 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-8053f4ce-0e76-4421-9c44-b861405b281c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:45.609444 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33394163-5c75-4b88-ab6a-f94d8da53814 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:45.614631 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33394163-5c75-4b88-ab6a-f94d8da53814 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:45.654168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-33394163-5c75-4b88-ab6a-f94d8da53814 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 17 16:40:45.654168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:45.654168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:45.654168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:45.654168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:45.654168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:45.654168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:45.654168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:45.654168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:45.654168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:45.654168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:45.654168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:45.654168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:40:45.654168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:40:45.654168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:40:45.654168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:40:45.655911 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:45.655911 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:45.655911 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:40:45.655911 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:40:45.655911 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:40:45.655911 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:40:45.655911 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:45.655911 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:45.655911 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:45.655911 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:45.655911 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:45.655911 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:45.655911 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:45.655911 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:45.655911 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:45.655911 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:45.655911 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:45.655911 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:45.658327 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:45.658327 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:45.658327 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:40:45.658327 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:40:45.658327 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:40:45.658327 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:40:45.658327 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:40:45.658327 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:40:45.658327 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 17 16:40:45.658327 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:45.658327 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 291/596] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:40:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:45.658327 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33394163-5c75-4b88-ab6a-f94d8da53814 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:45.726523 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:45.745904 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 306/597] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:45.750058 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053f4ce-0e76-4421-9c44-b861405b281c None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:45.884120 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33394163-5c75-4b88-ab6a-f94d8da53814 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:45.910063 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:45.946764 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053f4ce-0e76-4421-9c44-b861405b281c None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:45.950886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d181af6a-94dc-4ac8-a39c-bad723183097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:45.952388 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d181af6a-94dc-4ac8-a39c-bad723183097 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:45.953179 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d181af6a-94dc-4ac8-a39c-bad723183097 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:45.954586 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d181af6a-94dc-4ac8-a39c-bad723183097 None admin] PATH_INFO: `/v3/users` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:45.956360 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d181af6a-94dc-4ac8-a39c-bad723183097 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:45.957211 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d181af6a-94dc-4ac8-a39c-bad723183097 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:45.960014 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d181af6a-94dc-4ac8-a39c-bad723183097 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:45.996068 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 292/598] 10.222.0.40 () {56 vars in 1139 bytes} [Sun Jul 17 16:40:45 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:45.998410 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d181af6a-94dc-4ac8-a39c-bad723183097 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:46.017458 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2244d9b-cb49-42b5-ae79-832889432c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:46.018521 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2244d9b-cb49-42b5-ae79-832889432c15 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:46.018994 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2244d9b-cb49-42b5-ae79-832889432c15 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:46.019462 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2244d9b-cb49-42b5-ae79-832889432c15 None admin] PATH_INFO: `/v3/users` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:46.020974 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2244d9b-cb49-42b5-ae79-832889432c15 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:46.021972 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2244d9b-cb49-42b5-ae79-832889432c15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:46.024059 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2244d9b-cb49-42b5-ae79-832889432c15 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:46.061246 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 307/599] 10.222.0.40 () {56 vars in 1137 bytes} [Sun Jul 17 16:40:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:46.063360 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2244d9b-cb49-42b5-ae79-832889432c15 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:46.174203 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:46.198319 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d181af6a-94dc-4ac8-a39c-bad723183097 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:46.248556 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2244d9b-cb49-42b5-ae79-832889432c15 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:46.263705 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce19c5ae-1cb8-485d-9be1-80780e3a3599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:46.265616 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce19c5ae-1cb8-485d-9be1-80780e3a3599 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:46.266919 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce19c5ae-1cb8-485d-9be1-80780e3a3599 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:46.267646 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce19c5ae-1cb8-485d-9be1-80780e3a3599 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:46.287903 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce19c5ae-1cb8-485d-9be1-80780e3a3599 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:46.305010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-ce19c5ae-1cb8-485d-9be1-80780e3a3599 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:40:46.305010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:46.305010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:46.305010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:46.305010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:46.305010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:46.305010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:46.305010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:46.305010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:46.305010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:46.305010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:46.305010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:46.305010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:46.305010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:46.305010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:46.305010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:46.307211 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:46.307211 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:46.307211 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:46.307211 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:46.307211 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:46.307211 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:46.307211 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:46.307211 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:46.307211 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:46.307211 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:46.307211 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:46.307211 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:46.307211 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:46.307211 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:46.307211 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:46.307211 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:46.309074 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:46.309074 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:46.309074 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:46.309074 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:46.309074 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:46.309074 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:46.309074 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:46.309074 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:46.309074 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:46.309074 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:46.309074 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:40:46.309074 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:46.319355 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad3e06-beef-495a-803a-a1d5c8e26bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:46.319355 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad3e06-beef-495a-803a-a1d5c8e26bee None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:46.319355 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad3e06-beef-495a-803a-a1d5c8e26bee None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:46.319355 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad3e06-beef-495a-803a-a1d5c8e26bee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:46.320451 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 293/600] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 17 16:40:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:46.322245 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce19c5ae-1cb8-485d-9be1-80780e3a3599 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:46.339120 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad3e06-beef-495a-803a-a1d5c8e26bee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:46.341838 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad3e06-beef-495a-803a-a1d5c8e26bee None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:46.377963 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-c5ad3e06-beef-495a-803a-a1d5c8e26bee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:46.377963 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:46.377963 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:46.377963 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:46.377963 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:46.377963 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:46.377963 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:46.377963 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:46.377963 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:46.377963 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:46.377963 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:46.377963 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:46.377963 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:46.377963 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:46.377963 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:46.379944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:46.379944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:46.379944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:46.379944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:46.379944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:46.379944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:46.379944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:46.379944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:46.379944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:46.379944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:46.379944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:46.379944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:46.379944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:46.379944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:46.379944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:46.379944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:46.379944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:46.382081 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:46.382081 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:46.382081 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:46.382081 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:46.384040 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 308/601] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:46.389937 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad3e06-beef-495a-803a-a1d5c8e26bee None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:46.394359 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:46.540844 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce19c5ae-1cb8-485d-9be1-80780e3a3599 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:46.585267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d0682-84b1-4208-a631-4cf918fd0136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:46.586092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d0682-84b1-4208-a631-4cf918fd0136 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:46.586379 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d0682-84b1-4208-a631-4cf918fd0136 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:46.586663 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d0682-84b1-4208-a631-4cf918fd0136 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:46.600715 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d0682-84b1-4208-a631-4cf918fd0136 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:46.602363 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d0682-84b1-4208-a631-4cf918fd0136 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:46.610651 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad3e06-beef-495a-803a-a1d5c8e26bee None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:46.623098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-4e2d0682-84b1-4208-a631-4cf918fd0136 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:46.623098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:46.623098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:46.623098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:46.623098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:46.623098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:46.623098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:46.623098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:46.623098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:46.623098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:46.623098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:46.623098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:46.623098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:46.623098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:46.623098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:46.624894 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:46.624894 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:46.624894 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:46.624894 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:46.624894 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:46.624894 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:46.624894 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:46.624894 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:46.624894 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:46.624894 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:46.624894 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:46.624894 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:46.624894 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:46.624894 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:46.624894 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:46.624894 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:46.624894 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:46.627155 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:46.627155 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:46.627155 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:46.627155 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:46.627155 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 294/602] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:46.627155 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d0682-84b1-4208-a631-4cf918fd0136 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:46.634738 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:46.681005 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dcd05a-df86-43b0-bccd-26a06634d2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:46.683873 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dcd05a-df86-43b0-bccd-26a06634d2d6 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:46.684622 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dcd05a-df86-43b0-bccd-26a06634d2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:46.685309 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dcd05a-df86-43b0-bccd-26a06634d2d6 None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:46.686888 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dcd05a-df86-43b0-bccd-26a06634d2d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:46.688153 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dcd05a-df86-43b0-bccd-26a06634d2d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:46.691093 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dcd05a-df86-43b0-bccd-26a06634d2d6 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:46.708341 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 309/603] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 17 16:40:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:46.710431 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dcd05a-df86-43b0-bccd-26a06634d2d6 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:46.722843 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:46.743160 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d0682-84b1-4208-a631-4cf918fd0136 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:46.783617 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd219dda-ac85-4888-98bf-1629b0c9b7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:46.784140 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd219dda-ac85-4888-98bf-1629b0c9b7f3 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:46.784442 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd219dda-ac85-4888-98bf-1629b0c9b7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:46.784701 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd219dda-ac85-4888-98bf-1629b0c9b7f3 None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:46.785478 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd219dda-ac85-4888-98bf-1629b0c9b7f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:46.785892 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd219dda-ac85-4888-98bf-1629b0c9b7f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:46.787290 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd219dda-ac85-4888-98bf-1629b0c9b7f3 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:46.801299 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 295/604] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:46.802098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd219dda-ac85-4888-98bf-1629b0c9b7f3 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:46.917301 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd219dda-ac85-4888-98bf-1629b0c9b7f3 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:46.920404 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dcd05a-df86-43b0-bccd-26a06634d2d6 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:46.955557 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e4d0c-373e-4d7a-b7ca-2b993721a80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:46.956527 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e4d0c-373e-4d7a-b7ca-2b993721a80b None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:46.956692 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e4d0c-373e-4d7a-b7ca-2b993721a80b None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:46.957026 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e4d0c-373e-4d7a-b7ca-2b993721a80b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:46.969830 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0e4d0c-373e-4d7a-b7ca-2b993721a80b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:46.971405 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0e4d0c-373e-4d7a-b7ca-2b993721a80b None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:46.973568 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:46.990704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e775e293-8b35-45e4-8c23-cae83443ed4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:46.992383 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775e293-8b35-45e4-8c23-cae83443ed4f None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:46.993193 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775e293-8b35-45e4-8c23-cae83443ed4f None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:46.993667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-2e0e4d0c-373e-4d7a-b7ca-2b993721a80b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:46.993667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:46.993667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:46.993667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:46.993667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:46.993667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:46.993667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:46.993667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:46.993667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:46.993667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:46.993667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:46.993667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:46.993667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:46.993667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:46.993667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:46.995750 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:46.995750 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:46.995750 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:46.995750 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:46.995750 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:46.995750 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:46.995750 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:46.995750 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:46.995750 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:46.995750 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:46.995750 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:46.995750 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:46.995750 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:46.995750 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:46.995750 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:46.995750 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:46.995750 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:46.997922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:46.997922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:46.997922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:46.997922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:46.997922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 296/605] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:46.997922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e4d0c-373e-4d7a-b7ca-2b993721a80b None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:46.999905 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775e293-8b35-45e4-8c23-cae83443ed4f None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:47.002326 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e775e293-8b35-45e4-8c23-cae83443ed4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:47.003780 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e775e293-8b35-45e4-8c23-cae83443ed4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:47.006795 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e775e293-8b35-45e4-8c23-cae83443ed4f None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:47.032094 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 310/606] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:47.034051 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e775e293-8b35-45e4-8c23-cae83443ed4f None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:47.097832 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:47.115871 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e4d0c-373e-4d7a-b7ca-2b993721a80b None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:47.154806 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2af07f-39e2-4cdf-a9df-d8dbef4dc48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:47.155599 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2af07f-39e2-4cdf-a9df-d8dbef4dc48d None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:47.155886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2af07f-39e2-4cdf-a9df-d8dbef4dc48d None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:47.156226 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2af07f-39e2-4cdf-a9df-d8dbef4dc48d None admin] PATH_INFO: `/v3/projects/service` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:47.170492 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2af07f-39e2-4cdf-a9df-d8dbef4dc48d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:47.172103 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2af07f-39e2-4cdf-a9df-d8dbef4dc48d None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:47.193449 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-1c2af07f-39e2-4cdf-a9df-d8dbef4dc48d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:40:47.193449 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:47.193449 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:47.193449 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:47.193449 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:47.193449 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:47.193449 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:47.193449 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:47.193449 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:47.193449 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:47.193449 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:47.193449 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:47.193449 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:40:47.193449 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:40:47.193449 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:40:47.196079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:40:47.196079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:47.196079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:47.196079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:47.196079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:47.196079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:47.196079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:47.196079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:47.196079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:47.196079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:47.196079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:47.196079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:47.196079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:47.196079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:47.196079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:47.196079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:47.196079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:47.197896 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:40:47.197896 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:40:47.197896 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:40:47.197896 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:40:47.197896 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:40:47.197896 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:40:47.197896 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:40:47.197896 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:47.197896 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 297/607] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:40:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:47.197896 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2af07f-39e2-4cdf-a9df-d8dbef4dc48d None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:47.250028 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e775e293-8b35-45e4-8c23-cae83443ed4f None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:47.263158 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:47.315772 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2af07f-39e2-4cdf-a9df-d8dbef4dc48d None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:47.324981 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b10ca-4b89-4572-ba2b-f50c42784408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:47.326750 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b10ca-4b89-4572-ba2b-f50c42784408 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:47.327761 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b10ca-4b89-4572-ba2b-f50c42784408 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:47.329153 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b10ca-4b89-4572-ba2b-f50c42784408 None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:47.330887 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18b10ca-4b89-4572-ba2b-f50c42784408 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:47.331882 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18b10ca-4b89-4572-ba2b-f50c42784408 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:47.335017 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18b10ca-4b89-4572-ba2b-f50c42784408 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:47.356431 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e4b02a-fde1-4318-b151-17689a2ded7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:47.357219 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4b02a-fde1-4318-b151-17689a2ded7e None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:47.357473 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4b02a-fde1-4318-b151-17689a2ded7e None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:47.357759 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4b02a-fde1-4318-b151-17689a2ded7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:47.362612 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 311/608] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:47.364742 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b10ca-4b89-4572-ba2b-f50c42784408 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:47.366398 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e4b02a-fde1-4318-b151-17689a2ded7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:47.367904 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e4b02a-fde1-4318-b151-17689a2ded7e None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:47.385023 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-e8e4b02a-fde1-4318-b151-17689a2ded7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:40:47.385023 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:47.385023 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:47.385023 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:47.385023 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:47.385023 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:47.385023 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:47.385023 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:47.385023 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:47.385023 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:47.385023 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:47.385023 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:47.385023 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:40:47.385023 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:40:47.385023 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:40:47.387430 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:40:47.387430 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:47.387430 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:47.387430 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:47.387430 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:47.387430 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:47.387430 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:47.387430 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:47.387430 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:47.387430 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:47.387430 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:47.387430 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:47.387430 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:47.387430 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:47.387430 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:47.387430 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:47.387430 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:47.389580 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:40:47.389580 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:40:47.389580 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:40:47.389580 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:40:47.389580 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:40:47.389580 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:40:47.389580 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:40:47.389580 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:47.389580 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 298/609] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:40:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:47.389580 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e4b02a-fde1-4318-b151-17689a2ded7e None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:47.536737 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:47.553811 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b10ca-4b89-4572-ba2b-f50c42784408 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:47.564456 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e4b02a-fde1-4318-b151-17689a2ded7e None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:47.590743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:47.602155 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6440d847-2aa1-419c-bd08-079a252328a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:47.602839 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6440d847-2aa1-419c-bd08-079a252328a2 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:47.603131 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6440d847-2aa1-419c-bd08-079a252328a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:47.603409 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6440d847-2aa1-419c-bd08-079a252328a2 None admin] PATH_INFO: `/v3/projects` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:47.604351 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6440d847-2aa1-419c-bd08-079a252328a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:47.604672 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6440d847-2aa1-419c-bd08-079a252328a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:47.606204 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6440d847-2aa1-419c-bd08-079a252328a2 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:47.617678 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef46a5-8a04-4d03-adac-a4aa6691ccf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:47.617678 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef46a5-8a04-4d03-adac-a4aa6691ccf1 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:47.617678 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef46a5-8a04-4d03-adac-a4aa6691ccf1 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:47.617678 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef46a5-8a04-4d03-adac-a4aa6691ccf1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:47.622134 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 312/610] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:40:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:47.623281 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6440d847-2aa1-419c-bd08-079a252328a2 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:47.630961 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef46a5-8a04-4d03-adac-a4aa6691ccf1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:47.632670 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef46a5-8a04-4d03-adac-a4aa6691ccf1 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:47.652812 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-64ef46a5-8a04-4d03-adac-a4aa6691ccf1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 17 16:40:47.652812 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:47.652812 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:47.652812 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:47.652812 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:47.652812 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:47.652812 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:47.652812 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:47.652812 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:47.652812 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:47.652812 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:47.652812 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:47.652812 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:40:47.652812 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:40:47.652812 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:40:47.652812 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:40:47.654745 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:47.654745 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:47.654745 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:40:47.654745 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:40:47.654745 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:40:47.654745 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:40:47.654745 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:47.654745 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:47.654745 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:47.654745 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:47.654745 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:47.654745 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:47.654745 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:47.654745 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:47.654745 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:47.654745 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:47.654745 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:47.654745 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:47.657283 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:47.657283 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:47.657283 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:40:47.657283 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:40:47.657283 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:40:47.657283 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:40:47.657283 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:40:47.657283 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:40:47.657283 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 17 16:40:47.657283 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:47.662594 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 299/611] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:40:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:47.662594 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4f581-efa4-42ac-89e9-500e0d23bba1 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:47.662594 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4f581-efa4-42ac-89e9-500e0d23bba1 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:47.662594 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4f581-efa4-42ac-89e9-500e0d23bba1 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:47.666909 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 300/612] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:47.667833 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4f581-efa4-42ac-89e9-500e0d23bba1 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:47.769384 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6440d847-2aa1-419c-bd08-079a252328a2 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:47.807729 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e557b24c-90d0-406c-80ed-e55d7cac2c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:47.808203 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e557b24c-90d0-406c-80ed-e55d7cac2c28 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:47.808499 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e557b24c-90d0-406c-80ed-e55d7cac2c28 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:47.808806 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e557b24c-90d0-406c-80ed-e55d7cac2c28 None admin] PATH_INFO: `/v3/projects` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:47.809612 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e557b24c-90d0-406c-80ed-e55d7cac2c28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:47.809935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e557b24c-90d0-406c-80ed-e55d7cac2c28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:47.811415 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e557b24c-90d0-406c-80ed-e55d7cac2c28 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:47.825834 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 313/613] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:40:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:47.826729 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e557b24c-90d0-406c-80ed-e55d7cac2c28 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:47.843397 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:47.861554 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4f581-efa4-42ac-89e9-500e0d23bba1 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:47.929148 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f0ceb-7644-4096-bf5d-ac9eb1761dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:47.930694 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f0ceb-7644-4096-bf5d-ac9eb1761dcb None admin] REQUEST_METHOD: `PUT` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:47.931275 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f0ceb-7644-4096-bf5d-ac9eb1761dcb None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:47.931838 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f0ceb-7644-4096-bf5d-ac9eb1761dcb None admin] PATH_INFO: `/v3/projects/927057f9a5af4833aeac9992203a223c/users/9663edc8502b4aecb5317b24e4a5816b/roles/adb9c27c6e5246fdb0cf386358a822cd` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:47.946367 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e557b24c-90d0-406c-80ed-e55d7cac2c28 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:47.958922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f0ceb-7644-4096-bf5d-ac9eb1761dcb None admin] RBAC: Authorizing `identity:create_grant(project_id=927057f9a5af4833aeac9992203a223c, user_id=9663edc8502b4aecb5317b24e4a5816b, role_id=adb9c27c6e5246fdb0cf386358a822cd)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:47.961478 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f0ceb-7644-4096-bf5d-ac9eb1761dcb None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:47.998271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea41c16-8bbd-4704-b91c-f52a172cba2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:47.999332 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea41c16-8bbd-4704-b91c-f52a172cba2e None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:47.999842 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea41c16-8bbd-4704-b91c-f52a172cba2e None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:48.000902 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea41c16-8bbd-4704-b91c-f52a172cba2e None admin] PATH_INFO: `/v3/users` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:48.002789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea41c16-8bbd-4704-b91c-f52a172cba2e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:48.003446 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea41c16-8bbd-4704-b91c-f52a172cba2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:48.006030 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea41c16-8bbd-4704-b91c-f52a172cba2e None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:48.046027 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 314/614] 10.222.0.40 () {56 vars in 1135 bytes} [Sun Jul 17 16:40:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:48.046027 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d23374-d2fc-42df-aa2f-120d74169055 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:48.046027 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d23374-d2fc-42df-aa2f-120d74169055 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:48.046027 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d23374-d2fc-42df-aa2f-120d74169055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:48.064821 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 301/615] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 17 16:40:47 2022] PUT /identity/v3/projects/927057f9a5af4833aeac9992203a223c/users/9663edc8502b4aecb5317b24e4a5816b/roles/adb9c27c6e5246fdb0cf386358a822cd => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 17 16:40:48.066332 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f0ceb-7644-4096-bf5d-ac9eb1761dcb None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:48.075545 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-b8d23374-d2fc-42df-aa2f-120d74169055 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:48.076909 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-b8d23374-d2fc-42df-aa2f-120d74169055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:48.155170 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:48.183756 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:48.198240 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 315/616] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:48.198240 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d23374-d2fc-42df-aa2f-120d74169055 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:48.311628 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f0ceb-7644-4096-bf5d-ac9eb1761dcb None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:48.373089 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3edeb63-63e2-4d66-b954-e814aac028dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:48.373813 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d23374-d2fc-42df-aa2f-120d74169055 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:48.374919 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edeb63-63e2-4d66-b954-e814aac028dc None admin] REQUEST_METHOD: `PUT` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:48.375776 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edeb63-63e2-4d66-b954-e814aac028dc None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:48.376983 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edeb63-63e2-4d66-b954-e814aac028dc None admin] PATH_INFO: `/v3/projects/927057f9a5af4833aeac9992203a223c/users/0b7122fc744d446aa60b25844c68f15b/roles/adb9c27c6e5246fdb0cf386358a822cd` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:48.401736 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3edeb63-63e2-4d66-b954-e814aac028dc None admin] RBAC: Authorizing `identity:create_grant(project_id=927057f9a5af4833aeac9992203a223c, user_id=0b7122fc744d446aa60b25844c68f15b, role_id=adb9c27c6e5246fdb0cf386358a822cd)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:48.403476 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3edeb63-63e2-4d66-b954-e814aac028dc None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:48.422741 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26540c17-4401-42ae-987e-5992f613a2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:48.424130 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26540c17-4401-42ae-987e-5992f613a2b3 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:48.424944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26540c17-4401-42ae-987e-5992f613a2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:48.425257 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26540c17-4401-42ae-987e-5992f613a2b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:48.446248 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26540c17-4401-42ae-987e-5992f613a2b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:48.449004 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26540c17-4401-42ae-987e-5992f613a2b3 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:48.486011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-26540c17-4401-42ae-987e-5992f613a2b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:48.486011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:48.486011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:48.486011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:48.486011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:48.486011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:48.486011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:48.486011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:48.486011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:48.486011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:48.486011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:48.486011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:48.486011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:48.486011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:48.486011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:48.488091 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:48.488091 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:48.488091 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:48.488091 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:48.488091 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:48.488091 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:48.488091 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:48.488091 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:48.488091 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:48.488091 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:48.488091 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:48.488091 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:48.488091 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:48.488091 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:48.488091 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:48.488091 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:48.488091 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:48.490055 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:48.490055 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:48.490055 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:48.490055 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:48.490055 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 316/617] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:48.490626 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26540c17-4401-42ae-987e-5992f613a2b3 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:48.501369 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 302/618] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 17 16:40:48 2022] PUT /identity/v3/projects/927057f9a5af4833aeac9992203a223c/users/0b7122fc744d446aa60b25844c68f15b/roles/adb9c27c6e5246fdb0cf386358a822cd => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 17 16:40:48.503933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba981dd-7a19-433e-9b63-41ff1180571f None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:48.506346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba981dd-7a19-433e-9b63-41ff1180571f None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:48.506346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba981dd-7a19-433e-9b63-41ff1180571f None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:48.514124 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 303/619] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:48.514124 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba981dd-7a19-433e-9b63-41ff1180571f None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:48.615079 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:48.677569 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:48.724446 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26540c17-4401-42ae-987e-5992f613a2b3 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:48.743685 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba981dd-7a19-433e-9b63-41ff1180571f None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:48.806335 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1317ea3e-1a39-4233-913b-3458ec979bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:48.807716 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1317ea3e-1a39-4233-913b-3458ec979bb3 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:48.808124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1317ea3e-1a39-4233-913b-3458ec979bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:48.808584 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1317ea3e-1a39-4233-913b-3458ec979bb3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:48.819541 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9d7f65-0785-4e43-9277-e2548c3345e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:48.820979 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d7f65-0785-4e43-9277-e2548c3345e1 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:48.821628 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d7f65-0785-4e43-9277-e2548c3345e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:48.822233 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d7f65-0785-4e43-9277-e2548c3345e1 None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:48.823668 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d7f65-0785-4e43-9277-e2548c3345e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:48.824188 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d7f65-0785-4e43-9277-e2548c3345e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:48.826185 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d7f65-0785-4e43-9277-e2548c3345e1 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:48.829985 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1317ea3e-1a39-4233-913b-3458ec979bb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:48.846909 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 304/620] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:48.849089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-1317ea3e-1a39-4233-913b-3458ec979bb3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:40:48.849089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:48.849089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:48.849089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:48.849089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:48.849089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:48.849089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:48.849089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:48.849089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:48.849089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:48.849089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:48.849089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:48.849089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:48.849089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:48.849089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:48.849089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:48.851019 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:48.851019 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:48.851019 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:48.851019 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:48.851019 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:48.851019 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:48.851019 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:48.851019 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:48.851019 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:48.851019 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:48.851019 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:48.851019 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:48.851019 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:48.851019 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:48.851019 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:48.851019 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:48.852871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:48.852871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:48.852871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:48.852871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:48.852871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:48.852871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:48.852871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:48.852871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:48.852871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:48.852871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:48.852871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:40:48.852871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:48.854416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25549c47-f270-492c-bb37-5f0befb24d21 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:48.854416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25549c47-f270-492c-bb37-5f0befb24d21 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:48.854416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25549c47-f270-492c-bb37-5f0befb24d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:48.857717 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 317/621] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 17 16:40:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:48.866185 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1317ea3e-1a39-4233-913b-3458ec979bb3 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:48.889192 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-25549c47-f270-492c-bb37-5f0befb24d21 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:48.891749 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-25549c47-f270-492c-bb37-5f0befb24d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:49.024063 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 305/622] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:49.025955 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25549c47-f270-492c-bb37-5f0befb24d21 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:49.087301 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1317ea3e-1a39-4233-913b-3458ec979bb3 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:49.129238 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:49.150837 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:49.168021 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d9d8b-130c-483c-a90f-ebf498a46c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:49.169870 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d9d8b-130c-483c-a90f-ebf498a46c3a None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:49.179027 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d9d8b-130c-483c-a90f-ebf498a46c3a None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:49.179027 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d9d8b-130c-483c-a90f-ebf498a46c3a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:49.186851 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40d9d8b-130c-483c-a90f-ebf498a46c3a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:49.191097 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40d9d8b-130c-483c-a90f-ebf498a46c3a None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:49.223904 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-a40d9d8b-130c-483c-a90f-ebf498a46c3a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:40:49.223904 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:49.223904 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:49.223904 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:49.223904 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:49.223904 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:49.223904 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:49.223904 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:49.223904 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:49.223904 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:49.223904 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:49.223904 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:49.223904 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:40:49.223904 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:40:49.223904 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:40:49.225685 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:40:49.225685 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:49.225685 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:49.225685 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:49.225685 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:49.225685 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:49.225685 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:49.225685 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:49.225685 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:49.225685 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:49.225685 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:49.225685 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:49.225685 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:49.225685 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:49.225685 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:49.225685 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:49.225685 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:49.227356 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:40:49.227356 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:40:49.227356 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:40:49.227356 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:40:49.227356 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:40:49.227356 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:40:49.227356 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:40:49.227356 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:49.233252 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 318/623] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:40:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:49.235878 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d9d8b-130c-483c-a90f-ebf498a46c3a None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:49.247420 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25549c47-f270-492c-bb37-5f0befb24d21 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:49.314151 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cef9cc-bbe9-4f76-a735-4b9b262586ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:49.315186 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cef9cc-bbe9-4f76-a735-4b9b262586ee None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:49.315680 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cef9cc-bbe9-4f76-a735-4b9b262586ee None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:49.316175 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cef9cc-bbe9-4f76-a735-4b9b262586ee None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:49.317614 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cef9cc-bbe9-4f76-a735-4b9b262586ee None admin] RBAC: Adding query filter params (name=service) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:49.318163 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cef9cc-bbe9-4f76-a735-4b9b262586ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:49.321065 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cef9cc-bbe9-4f76-a735-4b9b262586ee None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:49.345149 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 306/624] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 17 16:40:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:49.346769 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cef9cc-bbe9-4f76-a735-4b9b262586ee None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:49.444722 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d9d8b-130c-483c-a90f-ebf498a46c3a None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:49.514982 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b7979-dc7d-4315-9294-b0ff1ceb697c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:49.517557 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b7979-dc7d-4315-9294-b0ff1ceb697c None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:49.517557 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b7979-dc7d-4315-9294-b0ff1ceb697c None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:49.518008 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b7979-dc7d-4315-9294-b0ff1ceb697c None admin] PATH_INFO: `/v3/roles/service` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:49.538946 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3b7979-dc7d-4315-9294-b0ff1ceb697c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:49.552371 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cef9cc-bbe9-4f76-a735-4b9b262586ee None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:49.555235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-ae3b7979-dc7d-4315-9294-b0ff1ceb697c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:40:49.555235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:49.555235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:49.555235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:49.555235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:49.555235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:49.555235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:49.555235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:49.555235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:49.555235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:49.555235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:49.555235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:49.555235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:49.555235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:49.555235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:49.555235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:49.556663 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:49.556663 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:49.556663 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:49.556663 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:49.556663 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:49.556663 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:49.556663 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:49.556663 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:49.556663 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:49.556663 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:49.556663 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:49.556663 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:49.556663 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:49.556663 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:49.556663 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:49.556663 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:49.558047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:49.558047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:49.558047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:49.558047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:49.558047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:49.558047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:49.558047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:49.558047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:49.558047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:49.558047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:49.558047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:40:49.558047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:49.565439 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 319/625] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 17 16:40:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:49.566977 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b7979-dc7d-4315-9294-b0ff1ceb697c None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:49.608019 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:49.624145 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8edafa7-238b-4eaa-ade2-c9b431a64129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:49.626293 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edafa7-238b-4eaa-ade2-c9b431a64129 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:49.627076 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edafa7-238b-4eaa-ade2-c9b431a64129 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:49.627837 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edafa7-238b-4eaa-ade2-c9b431a64129 None admin] PATH_INFO: `/v3/projects` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:49.629845 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8edafa7-238b-4eaa-ade2-c9b431a64129 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:49.630699 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8edafa7-238b-4eaa-ade2-c9b431a64129 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:49.631464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:49.633979 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8edafa7-238b-4eaa-ade2-c9b431a64129 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:49.659234 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 307/626] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:40:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:49.661237 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8edafa7-238b-4eaa-ade2-c9b431a64129 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:49.777072 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b7979-dc7d-4315-9294-b0ff1ceb697c None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:49.851108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3cb3f8-9d7d-4826-a96c-3b94f339df7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:49.852615 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3cb3f8-9d7d-4826-a96c-3b94f339df7d None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:49.853079 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3cb3f8-9d7d-4826-a96c-3b94f339df7d None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:49.853636 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3cb3f8-9d7d-4826-a96c-3b94f339df7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:49.873715 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8edafa7-238b-4eaa-ade2-c9b431a64129 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:49.876381 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3cb3f8-9d7d-4826-a96c-3b94f339df7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:49.879379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3cb3f8-9d7d-4826-a96c-3b94f339df7d None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:49.916290 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-3b3cb3f8-9d7d-4826-a96c-3b94f339df7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:49.916290 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:49.916290 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:49.916290 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:49.916290 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:49.916290 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:49.916290 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:49.916290 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:49.916290 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:49.916290 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:49.916290 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:49.916290 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:49.916290 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:49.916290 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:49.916290 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:49.918061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:49.918061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:49.918061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:49.918061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:49.918061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:49.918061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:49.918061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:49.918061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:49.918061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:49.918061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:49.918061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:49.918061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:49.918061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:49.918061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:49.918061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:49.918061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:49.918061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:49.919812 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:49.919812 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:49.919812 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:49.919812 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:49.925308 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 320/627] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:49.926939 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3cb3f8-9d7d-4826-a96c-3b94f339df7d None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:49.950726 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194b4e9-e4b3-444c-9af8-7411baf1e4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:49.952347 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194b4e9-e4b3-444c-9af8-7411baf1e4be None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:49.953710 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194b4e9-e4b3-444c-9af8-7411baf1e4be None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:49.954499 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194b4e9-e4b3-444c-9af8-7411baf1e4be None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:49.956226 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194b4e9-e4b3-444c-9af8-7411baf1e4be None admin] RBAC: Adding query filter params (name=service) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:49.957100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194b4e9-e4b3-444c-9af8-7411baf1e4be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:49.959967 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194b4e9-e4b3-444c-9af8-7411baf1e4be None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:49.977259 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 308/628] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 17 16:40:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:49.979220 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194b4e9-e4b3-444c-9af8-7411baf1e4be None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:50.077556 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:50.115973 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:50.149611 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3cb3f8-9d7d-4826-a96c-3b94f339df7d None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:50.184452 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194b4e9-e4b3-444c-9af8-7411baf1e4be None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:50.245528 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661615ad-103d-4209-8c03-93201cf76211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:50.247190 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661615ad-103d-4209-8c03-93201cf76211 None admin] REQUEST_METHOD: `PUT` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:50.247936 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661615ad-103d-4209-8c03-93201cf76211 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:50.248431 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661615ad-103d-4209-8c03-93201cf76211 None admin] PATH_INFO: `/v3/projects/927057f9a5af4833aeac9992203a223c/users/69482eeb1e8547aabca9f036b5c34f13/roles/adb9c27c6e5246fdb0cf386358a822cd` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:50.259431 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d983a7-7cee-4b9d-93e3-1681a8f29bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:50.261119 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d983a7-7cee-4b9d-93e3-1681a8f29bf1 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:50.261911 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d983a7-7cee-4b9d-93e3-1681a8f29bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:50.262708 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d983a7-7cee-4b9d-93e3-1681a8f29bf1 None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:50.264431 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d983a7-7cee-4b9d-93e3-1681a8f29bf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:50.265354 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d983a7-7cee-4b9d-93e3-1681a8f29bf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:50.268940 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d983a7-7cee-4b9d-93e3-1681a8f29bf1 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:50.283620 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661615ad-103d-4209-8c03-93201cf76211 None admin] RBAC: Authorizing `identity:create_grant(project_id=927057f9a5af4833aeac9992203a223c, user_id=69482eeb1e8547aabca9f036b5c34f13, role_id=adb9c27c6e5246fdb0cf386358a822cd)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:50.286881 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661615ad-103d-4209-8c03-93201cf76211 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:50.306752 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 309/629] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:50.306752 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d983a7-7cee-4b9d-93e3-1681a8f29bf1 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:50.377223 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 321/630] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 17 16:40:49 2022] PUT /identity/v3/projects/927057f9a5af4833aeac9992203a223c/users/69482eeb1e8547aabca9f036b5c34f13/roles/adb9c27c6e5246fdb0cf386358a822cd => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 17 16:40:50.378683 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661615ad-103d-4209-8c03-93201cf76211 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:50.543715 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d983a7-7cee-4b9d-93e3-1681a8f29bf1 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:50.565432 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:50.619013 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e10dc-09a5-4495-bd69-8068017c4a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:50.620270 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e10dc-09a5-4495-bd69-8068017c4a07 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:50.620833 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e10dc-09a5-4495-bd69-8068017c4a07 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:50.621285 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e10dc-09a5-4495-bd69-8068017c4a07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:50.659825 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e10dc-09a5-4495-bd69-8068017c4a07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:50.662346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e10dc-09a5-4495-bd69-8068017c4a07 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:50.666692 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661615ad-103d-4209-8c03-93201cf76211 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:50.704943 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-6c3e10dc-09a5-4495-bd69-8068017c4a07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:50.704943 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:50.704943 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:50.704943 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:50.704943 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:50.704943 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:50.704943 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:50.704943 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:50.704943 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:50.704943 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:50.704943 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:50.704943 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:50.704943 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:50.704943 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:50.704943 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:50.706513 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:50.706513 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:50.706513 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:50.706513 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:50.706513 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:50.706513 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:50.706513 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:50.706513 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:50.706513 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:50.706513 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:50.706513 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:50.706513 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:50.706513 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:50.706513 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:50.706513 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:50.706513 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:50.706513 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:50.708194 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:50.708194 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:50.708194 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:50.708194 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:50.708194 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 310/631] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:50.717602 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e10dc-09a5-4495-bd69-8068017c4a07 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:50.746215 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b46a6-c0a9-4cee-af93-d9ed8217f6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:50.748142 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b46a6-c0a9-4cee-af93-d9ed8217f6a7 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:50.749085 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b46a6-c0a9-4cee-af93-d9ed8217f6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:50.749903 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b46a6-c0a9-4cee-af93-d9ed8217f6a7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:50.755017 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:50.768272 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b46a6-c0a9-4cee-af93-d9ed8217f6a7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:50.771548 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b46a6-c0a9-4cee-af93-d9ed8217f6a7 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:50.800302 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-e99b46a6-c0a9-4cee-af93-d9ed8217f6a7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 17 16:40:50.800302 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:50.800302 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:50.800302 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:50.800302 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:50.800302 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:50.800302 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:50.800302 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:50.800302 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:50.800302 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:50.800302 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:50.800302 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:50.800302 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:40:50.800302 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:40:50.800302 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:40:50.800302 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:40:50.801344 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:50.801344 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:50.801344 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:40:50.801344 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:40:50.801344 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:40:50.801344 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:40:50.801344 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:50.801344 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:50.801344 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:50.801344 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:50.801344 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:50.801344 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:50.801344 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:50.801344 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:50.801344 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:50.801344 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:50.801344 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:50.801344 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:50.802260 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:50.802260 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:50.802260 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:40:50.802260 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:40:50.802260 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:40:50.802260 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:40:50.802260 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:40:50.802260 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:40:50.802260 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 17 16:40:50.802260 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:50.805392 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 322/632] 10.222.0.40 () {56 vars in 1093 bytes} [Sun Jul 17 16:40:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 428 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:50.811275 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b46a6-c0a9-4cee-af93-d9ed8217f6a7 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:50.922551 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b46a6-c0a9-4cee-af93-d9ed8217f6a7 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:50.958261 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e10dc-09a5-4495-bd69-8068017c4a07 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:50.961066 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece5bd5f-31ff-4170-b63a-96bf81e6570b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:50.963044 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5bd5f-31ff-4170-b63a-96bf81e6570b None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:50.963535 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5bd5f-31ff-4170-b63a-96bf81e6570b None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:50.963991 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5bd5f-31ff-4170-b63a-96bf81e6570b None admin] PATH_INFO: `/v3/users` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:50.965125 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece5bd5f-31ff-4170-b63a-96bf81e6570b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:50.965633 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece5bd5f-31ff-4170-b63a-96bf81e6570b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:50.967302 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece5bd5f-31ff-4170-b63a-96bf81e6570b None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:50.988685 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 323/633] 10.222.0.40 () {56 vars in 1133 bytes} [Sun Jul 17 16:40:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:50.994629 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece5bd5f-31ff-4170-b63a-96bf81e6570b None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:51.061295 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c998f54-9755-429e-b63b-8069ff3bf910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:51.062288 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c998f54-9755-429e-b63b-8069ff3bf910 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:51.063970 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c998f54-9755-429e-b63b-8069ff3bf910 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:51.064491 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c998f54-9755-429e-b63b-8069ff3bf910 None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:51.065461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c998f54-9755-429e-b63b-8069ff3bf910 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:51.065952 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c998f54-9755-429e-b63b-8069ff3bf910 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:51.067724 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c998f54-9755-429e-b63b-8069ff3bf910 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:51.086283 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 311/634] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:51.093884 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c998f54-9755-429e-b63b-8069ff3bf910 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:51.094632 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:51.143694 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:51.222160 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece5bd5f-31ff-4170-b63a-96bf81e6570b None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:51.291197 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07605ea-2a73-4e7c-bac0-23c7cf35c90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:51.292274 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07605ea-2a73-4e7c-bac0-23c7cf35c90b None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:51.292799 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07605ea-2a73-4e7c-bac0-23c7cf35c90b None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:51.293278 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07605ea-2a73-4e7c-bac0-23c7cf35c90b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:51.315914 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c998f54-9755-429e-b63b-8069ff3bf910 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:51.373629 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07605ea-2a73-4e7c-bac0-23c7cf35c90b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:51.375845 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07605ea-2a73-4e7c-bac0-23c7cf35c90b None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:51.418356 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f77bfd5-0317-4b5f-8bff-4463d37e7794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:51.419759 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f77bfd5-0317-4b5f-8bff-4463d37e7794 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:51.420277 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f77bfd5-0317-4b5f-8bff-4463d37e7794 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:51.420819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f77bfd5-0317-4b5f-8bff-4463d37e7794 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:51.445507 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f77bfd5-0317-4b5f-8bff-4463d37e7794 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:51.449571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f77bfd5-0317-4b5f-8bff-4463d37e7794 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:51.470807 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:51.481286 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-7f77bfd5-0317-4b5f-8bff-4463d37e7794 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 17 16:40:51.481286 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:51.481286 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:51.481286 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:51.481286 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:51.481286 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:51.481286 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:51.481286 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:51.481286 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:51.481286 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:51.481286 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:51.481286 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:51.481286 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:40:51.481286 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:40:51.481286 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:40:51.481286 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:40:51.498707 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:51.498707 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:51.498707 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:40:51.498707 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:40:51.498707 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:40:51.498707 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:40:51.498707 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:51.498707 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:51.498707 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:51.498707 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:51.498707 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:51.498707 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:51.498707 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:51.498707 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:51.498707 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:51.498707 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:51.498707 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:51.498707 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:51.501166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:51.501166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:51.501166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:40:51.501166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:40:51.501166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:40:51.501166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:40:51.501166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:40:51.501166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:40:51.501166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 17 16:40:51.501166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:51.501166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 312/635] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:40:51 2022] GET /identity/v3/users/glance => generated 84 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:51.501166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2776e069-9773-43c7-a481-51a115216d1c None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:51.501166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2776e069-9773-43c7-a481-51a115216d1c None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:51.502562 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2776e069-9773-43c7-a481-51a115216d1c None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:51.504000 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 313/636] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:51.506001 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2776e069-9773-43c7-a481-51a115216d1c None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:51.592940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 324/637] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:40:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 17 16:40:51.596826 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3e97f-65de-4df6-a8c0-8391e8bb6f95 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:51.597357 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3e97f-65de-4df6-a8c0-8391e8bb6f95 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:51.599045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3e97f-65de-4df6-a8c0-8391e8bb6f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:51.635372 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-4fd3e97f-65de-4df6-a8c0-8391e8bb6f95 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:51.637747 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-4fd3e97f-65de-4df6-a8c0-8391e8bb6f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:51.685486 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:51.732240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2776e069-9773-43c7-a481-51a115216d1c None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:51.773312 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 325/638] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:51.774917 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd3e97f-65de-4df6-a8c0-8391e8bb6f95 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:51.802405 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb10d6-a405-4507-be51-f15d90bcd2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:51.803613 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb10d6-a405-4507-be51-f15d90bcd2c0 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:51.804123 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb10d6-a405-4507-be51-f15d90bcd2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:51.804633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb10d6-a405-4507-be51-f15d90bcd2c0 None admin] PATH_INFO: `/v3/users` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:51.806235 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb10d6-a405-4507-be51-f15d90bcd2c0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:51.806777 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb10d6-a405-4507-be51-f15d90bcd2c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:51.809664 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb10d6-a405-4507-be51-f15d90bcd2c0 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:51.842886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 314/639] 10.222.0.40 () {56 vars in 1137 bytes} [Sun Jul 17 16:40:51 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:51.846197 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d9a05-9710-429c-81ce-d1dec5f1f0b3 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:51.846705 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d9a05-9710-429c-81ce-d1dec5f1f0b3 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:51.847199 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d9a05-9710-429c-81ce-d1dec5f1f0b3 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:51.853128 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 315/640] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:51 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:51.854734 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d9a05-9710-429c-81ce-d1dec5f1f0b3 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:51.897550 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:51.927584 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd3e97f-65de-4df6-a8c0-8391e8bb6f95 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:51.995821 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eaaf12-901b-4d66-b50f-9c93b38212be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:51.997151 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eaaf12-901b-4d66-b50f-9c93b38212be None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:51.997629 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eaaf12-901b-4d66-b50f-9c93b38212be None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:51.998092 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eaaf12-901b-4d66-b50f-9c93b38212be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:52.018366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d9a05-9710-429c-81ce-d1dec5f1f0b3 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:52.019133 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eaaf12-901b-4d66-b50f-9c93b38212be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:52.021352 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eaaf12-901b-4d66-b50f-9c93b38212be None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:52.058860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-07eaaf12-901b-4d66-b50f-9c93b38212be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:52.058860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:52.058860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:52.058860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:52.058860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:52.058860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:52.058860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:52.058860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:52.058860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:52.058860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:52.058860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:52.058860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:52.058860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:52.058860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:52.058860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:52.060454 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:52.060454 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:52.060454 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:52.060454 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:52.060454 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:52.060454 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:52.060454 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:52.060454 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:52.060454 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:52.060454 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:52.060454 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:52.060454 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:52.060454 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:52.060454 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:52.060454 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:52.060454 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:52.060454 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:52.062240 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:52.062240 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:52.062240 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:52.062240 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:52.062240 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 326/641] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:52.064594 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eaaf12-901b-4d66-b50f-9c93b38212be None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:52.069488 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9468ef9d-fdc2-4478-a1d1-6974fdc111f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:52.070322 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468ef9d-fdc2-4478-a1d1-6974fdc111f1 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:52.070599 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468ef9d-fdc2-4478-a1d1-6974fdc111f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:52.070886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468ef9d-fdc2-4478-a1d1-6974fdc111f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:52.081706 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:52.086569 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9468ef9d-fdc2-4478-a1d1-6974fdc111f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:52.095027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-9468ef9d-fdc2-4478-a1d1-6974fdc111f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:40:52.095027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:52.095027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:52.095027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:52.095027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:52.095027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:52.095027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:52.095027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:52.095027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:52.095027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:52.095027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:52.095027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:52.095027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:52.095027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:52.095027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:52.095027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:52.097314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:52.097314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:52.097314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:52.097314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:52.097314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:52.097314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:52.097314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:52.097314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:52.097314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:52.097314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:52.097314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:52.097314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:52.097314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:52.097314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:52.097314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:52.097314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:52.098944 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:52.098944 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:52.098944 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:52.098944 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:52.098944 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:52.098944 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:52.098944 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:52.098944 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:52.098944 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:52.098944 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:52.098944 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:40:52.098944 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:52.098944 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 316/642] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 17 16:40:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:52.098944 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a403e2d-fcc7-466b-b7b9-268ff89973ad None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:52.102875 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a403e2d-fcc7-466b-b7b9-268ff89973ad None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:52.102875 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a403e2d-fcc7-466b-b7b9-268ff89973ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:52.125924 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-2a403e2d-fcc7-466b-b7b9-268ff89973ad None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:52.127271 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-2a403e2d-fcc7-466b-b7b9-268ff89973ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:52.258609 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 317/643] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:52.260234 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a403e2d-fcc7-466b-b7b9-268ff89973ad None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:52.261561 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eaaf12-901b-4d66-b50f-9c93b38212be None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:52.332234 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1bd2d-3583-4263-a7fd-339a6380eda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:52.333205 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1bd2d-3583-4263-a7fd-339a6380eda8 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:52.334227 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1bd2d-3583-4263-a7fd-339a6380eda8 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:52.334643 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1bd2d-3583-4263-a7fd-339a6380eda8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:52.339058 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:52.414234 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a403e2d-fcc7-466b-b7b9-268ff89973ad None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:52.418687 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a1bd2d-3583-4263-a7fd-339a6380eda8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:52.420416 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a1bd2d-3583-4263-a7fd-339a6380eda8 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:52.448991 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:52.461087 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa3882a-a009-483a-b2c6-a2bf405ba59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:52.461960 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa3882a-a009-483a-b2c6-a2bf405ba59f None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:52.462370 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa3882a-a009-483a-b2c6-a2bf405ba59f None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:52.462839 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa3882a-a009-483a-b2c6-a2bf405ba59f None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:52.463694 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa3882a-a009-483a-b2c6-a2bf405ba59f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:52.464176 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa3882a-a009-483a-b2c6-a2bf405ba59f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:52.466152 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa3882a-a009-483a-b2c6-a2bf405ba59f None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:52.488572 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 318/644] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:52.488572 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa3882a-a009-483a-b2c6-a2bf405ba59f None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:52.566756 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 327/645] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:40:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 17 16:40:52.567949 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1bd2d-3583-4263-a7fd-339a6380eda8 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:52.614179 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:52.642633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa3882a-a009-483a-b2c6-a2bf405ba59f None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:52.688164 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86266240-1063-4aa0-9fe7-1ed3dca793a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:52.689027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86266240-1063-4aa0-9fe7-1ed3dca793a6 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:52.689817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86266240-1063-4aa0-9fe7-1ed3dca793a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:52.689976 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86266240-1063-4aa0-9fe7-1ed3dca793a6 None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:52.691143 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86266240-1063-4aa0-9fe7-1ed3dca793a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:52.691582 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86266240-1063-4aa0-9fe7-1ed3dca793a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:52.693691 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86266240-1063-4aa0-9fe7-1ed3dca793a6 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:52.700351 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1bd2d-3583-4263-a7fd-339a6380eda8 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:52.713332 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 319/646] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 17 16:40:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:52.713332 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86266240-1063-4aa0-9fe7-1ed3dca793a6 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:52.756545 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fad003-e22c-4f0e-855a-04731d4763f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:52.757675 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fad003-e22c-4f0e-855a-04731d4763f0 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:52.758083 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fad003-e22c-4f0e-855a-04731d4763f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:52.758487 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fad003-e22c-4f0e-855a-04731d4763f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:52.773500 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fad003-e22c-4f0e-855a-04731d4763f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:52.779725 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:52.784052 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-13fad003-e22c-4f0e-855a-04731d4763f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:40:52.784052 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:52.784052 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:52.784052 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:52.784052 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:52.784052 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:52.784052 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:52.784052 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:52.784052 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:52.784052 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:52.784052 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:52.784052 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:52.784052 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:52.784052 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:52.784052 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:52.784052 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:52.786211 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:52.786211 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:52.786211 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:52.786211 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:52.786211 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:52.786211 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:52.786211 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:52.786211 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:52.786211 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:52.786211 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:52.786211 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:52.786211 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:52.786211 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:52.786211 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:52.786211 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:52.786211 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:52.787899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:52.787899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:52.787899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:52.787899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:52.787899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:52.787899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:52.787899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:52.787899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:52.787899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:52.787899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:52.787899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:40:52.787899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:52.787899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 328/647] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 17 16:40:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:52.787899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fad003-e22c-4f0e-855a-04731d4763f0 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:52.866474 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86266240-1063-4aa0-9fe7-1ed3dca793a6 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:52.916119 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd71b0-73b6-4a75-99d0-3ee29ea8b801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:52.917347 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd71b0-73b6-4a75-99d0-3ee29ea8b801 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:52.917759 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd71b0-73b6-4a75-99d0-3ee29ea8b801 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:52.918227 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd71b0-73b6-4a75-99d0-3ee29ea8b801 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:52.921139 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:52.929317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd71b0-73b6-4a75-99d0-3ee29ea8b801 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:52.930771 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fad003-e22c-4f0e-855a-04731d4763f0 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:52.930885 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd71b0-73b6-4a75-99d0-3ee29ea8b801 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:52.949773 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-a3bd71b0-73b6-4a75-99d0-3ee29ea8b801 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:40:52.949773 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:52.949773 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:52.949773 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:52.949773 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:52.949773 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:52.949773 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:52.949773 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:52.949773 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:52.949773 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:52.949773 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:52.949773 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:52.949773 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:40:52.949773 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:40:52.949773 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:40:52.950727 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:40:52.950727 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:52.950727 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:52.950727 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:52.950727 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:52.950727 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:52.950727 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:52.950727 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:52.950727 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:52.950727 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:52.950727 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:52.950727 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:52.950727 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:52.950727 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:52.950727 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:52.950727 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:52.950727 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:52.951613 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:40:52.951613 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:40:52.951613 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:40:52.951613 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:40:52.951613 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:40:52.951613 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:40:52.951613 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:40:52.951613 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:52.951613 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 320/648] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:40:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:52.952506 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd71b0-73b6-4a75-99d0-3ee29ea8b801 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:52.976344 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0c3fb-49e0-4c27-96e5-579f6a147e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:52.977585 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0c3fb-49e0-4c27-96e5-579f6a147e40 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:52.978061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0c3fb-49e0-4c27-96e5-579f6a147e40 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:52.978492 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0c3fb-49e0-4c27-96e5-579f6a147e40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:52.991615 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a0c3fb-49e0-4c27-96e5-579f6a147e40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:52.993536 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a0c3fb-49e0-4c27-96e5-579f6a147e40 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:53.014065 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-36a0c3fb-49e0-4c27-96e5-579f6a147e40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:53.014065 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:53.014065 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:53.014065 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:53.014065 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:53.014065 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:53.014065 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:53.014065 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:53.014065 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:53.014065 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:53.014065 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:53.014065 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:53.014065 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:53.014065 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:53.014065 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:53.015126 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:53.015126 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:53.015126 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:53.015126 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:53.015126 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:53.015126 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:53.015126 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:53.015126 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:53.015126 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:53.015126 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:53.015126 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:53.015126 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:53.015126 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:53.015126 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:53.015126 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:53.015126 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:53.015126 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:53.016364 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:53.016364 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:53.016364 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:53.016364 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:53.018053 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 329/649] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:53.019095 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0c3fb-49e0-4c27-96e5-579f6a147e40 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:53.094311 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd71b0-73b6-4a75-99d0-3ee29ea8b801 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:53.112413 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:53.153023 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffae60d-c019-49ce-b099-e7c3a6cf935f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:53.154150 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffae60d-c019-49ce-b099-e7c3a6cf935f None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:53.154590 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffae60d-c019-49ce-b099-e7c3a6cf935f None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:53.155041 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffae60d-c019-49ce-b099-e7c3a6cf935f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:53.178072 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffae60d-c019-49ce-b099-e7c3a6cf935f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:53.180592 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffae60d-c019-49ce-b099-e7c3a6cf935f None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:53.180701 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0c3fb-49e0-4c27-96e5-579f6a147e40 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:53.205964 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-0ffae60d-c019-49ce-b099-e7c3a6cf935f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:53.205964 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:53.205964 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:53.205964 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:53.205964 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:53.205964 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:53.205964 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:53.205964 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:53.205964 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:53.205964 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:53.205964 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:53.205964 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:53.205964 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:53.205964 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:53.205964 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:53.207701 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:53.207701 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:53.207701 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:53.207701 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:53.207701 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:53.207701 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:53.207701 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:53.207701 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:53.207701 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:53.207701 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:53.207701 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:53.207701 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:53.207701 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:53.207701 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:53.207701 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:53.207701 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:53.207701 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:53.210691 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:53.210691 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:53.210691 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:53.210691 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:53.210691 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 321/650] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:53.210691 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffae60d-c019-49ce-b099-e7c3a6cf935f None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:53.236100 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da07709b-8f69-4c2a-8e93-95799b37c829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:53.237104 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07709b-8f69-4c2a-8e93-95799b37c829 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:53.237515 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07709b-8f69-4c2a-8e93-95799b37c829 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:53.237911 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07709b-8f69-4c2a-8e93-95799b37c829 None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:53.239129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07709b-8f69-4c2a-8e93-95799b37c829 None admin] RBAC: Adding query filter params (name=service) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:53.239578 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07709b-8f69-4c2a-8e93-95799b37c829 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:53.241415 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07709b-8f69-4c2a-8e93-95799b37c829 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:53.255490 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 330/651] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 17 16:40:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:53.258397 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da07709b-8f69-4c2a-8e93-95799b37c829 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:53.280044 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:53.352097 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffae60d-c019-49ce-b099-e7c3a6cf935f None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:53.395476 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da07709b-8f69-4c2a-8e93-95799b37c829 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:53.398033 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e9dc8-93b5-49b4-8a40-8791a556ba73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:53.399074 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e9dc8-93b5-49b4-8a40-8791a556ba73 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:53.399551 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e9dc8-93b5-49b4-8a40-8791a556ba73 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:53.400009 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e9dc8-93b5-49b4-8a40-8791a556ba73 None admin] PATH_INFO: `/v3/projects` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:53.400961 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64e9dc8-93b5-49b4-8a40-8791a556ba73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:53.401324 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64e9dc8-93b5-49b4-8a40-8791a556ba73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:53.402844 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64e9dc8-93b5-49b4-8a40-8791a556ba73 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:53.418134 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 322/652] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:40:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:53.419492 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e9dc8-93b5-49b4-8a40-8791a556ba73 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:53.450700 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235246cb-b0d2-4029-8408-782be7236e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:53.451522 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235246cb-b0d2-4029-8408-782be7236e52 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:53.451815 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235246cb-b0d2-4029-8408-782be7236e52 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:53.452100 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235246cb-b0d2-4029-8408-782be7236e52 None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:53.452970 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235246cb-b0d2-4029-8408-782be7236e52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:53.453847 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235246cb-b0d2-4029-8408-782be7236e52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:53.455226 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:53.455388 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235246cb-b0d2-4029-8408-782be7236e52 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:53.469688 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 331/653] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:53.470584 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235246cb-b0d2-4029-8408-782be7236e52 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:53.555496 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:53.569121 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e9dc8-93b5-49b4-8a40-8791a556ba73 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:53.605919 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235246cb-b0d2-4029-8408-782be7236e52 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:53.625619 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c340533-37f5-43eb-879e-fcec42ecb3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:53.626513 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c340533-37f5-43eb-879e-fcec42ecb3cd None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:53.627236 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c340533-37f5-43eb-879e-fcec42ecb3cd None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:53.627726 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c340533-37f5-43eb-879e-fcec42ecb3cd None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:53.629217 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c340533-37f5-43eb-879e-fcec42ecb3cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:53.629738 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c340533-37f5-43eb-879e-fcec42ecb3cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:53.632226 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c340533-37f5-43eb-879e-fcec42ecb3cd None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:53.658324 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 323/654] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:53.658324 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c340533-37f5-43eb-879e-fcec42ecb3cd None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:53.671296 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498e37f-a210-4717-a7c9-203dcaec03dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:53.672531 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498e37f-a210-4717-a7c9-203dcaec03dd None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:53.673208 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498e37f-a210-4717-a7c9-203dcaec03dd None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:53.673672 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498e37f-a210-4717-a7c9-203dcaec03dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:53.691943 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b498e37f-a210-4717-a7c9-203dcaec03dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:53.694875 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b498e37f-a210-4717-a7c9-203dcaec03dd None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:53.725747 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-b498e37f-a210-4717-a7c9-203dcaec03dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:53.725747 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:53.725747 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:53.725747 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:53.725747 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:53.725747 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:53.725747 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:53.725747 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:53.725747 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:53.725747 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:53.725747 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:53.725747 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:53.725747 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:53.725747 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:53.725747 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:53.726708 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:53.726708 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:53.726708 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:53.726708 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:53.726708 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:53.726708 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:53.726708 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:53.726708 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:53.726708 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:53.726708 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:53.726708 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:53.726708 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:53.726708 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:53.726708 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:53.726708 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:53.726708 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:53.726708 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:53.727574 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:53.727574 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:53.727574 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:53.727574 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:53.727574 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 332/655] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:53.728849 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498e37f-a210-4717-a7c9-203dcaec03dd None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:53.798525 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:53.821588 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c340533-37f5-43eb-879e-fcec42ecb3cd None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:53.865297 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b3713-909f-43d1-be34-61d8f5838b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:53.865973 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b3713-909f-43d1-be34-61d8f5838b3f None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:53.866270 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b3713-909f-43d1-be34-61d8f5838b3f None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:53.866550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b3713-909f-43d1-be34-61d8f5838b3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:53.867425 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b3713-909f-43d1-be34-61d8f5838b3f None admin] RBAC: Adding query filter params (role.id=adb9c27c6e5246fdb0cf386358a822cd, scope.project.id=927057f9a5af4833aeac9992203a223c, user.id=03244087f1a34bddb11ea46aba11a148) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:53.867741 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b3713-909f-43d1-be34-61d8f5838b3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:53.869260 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b3713-909f-43d1-be34-61d8f5838b3f None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:53.874506 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498e37f-a210-4717-a7c9-203dcaec03dd None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:53.883902 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 324/656] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:40:53 2022] GET /identity/v3/role_assignments?user.id=03244087f1a34bddb11ea46aba11a148&scope.project.id=927057f9a5af4833aeac9992203a223c&role.id=adb9c27c6e5246fdb0cf386358a822cd => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:53.883902 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b3713-909f-43d1-be34-61d8f5838b3f None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:53.919714 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:53.944973 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25036780-1f53-4b92-8601-d134d284e50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:53.947027 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25036780-1f53-4b92-8601-d134d284e50f None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:53.947504 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25036780-1f53-4b92-8601-d134d284e50f None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:53.948316 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25036780-1f53-4b92-8601-d134d284e50f None admin] PATH_INFO: `/v3/projects/service` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:53.960238 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25036780-1f53-4b92-8601-d134d284e50f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:53.962023 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25036780-1f53-4b92-8601-d134d284e50f None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:53.982568 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-25036780-1f53-4b92-8601-d134d284e50f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:40:53.982568 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:53.982568 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:53.982568 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:53.982568 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:53.982568 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:53.982568 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:53.982568 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:53.982568 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:53.982568 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:53.982568 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:53.982568 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:53.982568 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:40:53.982568 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:40:53.982568 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:40:53.984538 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:40:53.984538 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:53.984538 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:53.984538 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:53.984538 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:53.984538 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:53.984538 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:53.984538 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:53.984538 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:53.984538 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:53.984538 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:53.984538 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:53.984538 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:53.984538 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:53.984538 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:53.984538 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:53.984538 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:53.986850 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:40:53.986850 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:40:53.986850 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:40:53.986850 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:40:53.986850 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:40:53.986850 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:40:53.986850 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:40:53.986850 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:53.989261 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 333/657] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:40:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:53.992957 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25036780-1f53-4b92-8601-d134d284e50f None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:54.085829 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b3713-909f-43d1-be34-61d8f5838b3f None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:54.153102 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15fc94-f615-4150-b986-06ee5feaec44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:54.154437 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15fc94-f615-4150-b986-06ee5feaec44 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:54.154911 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15fc94-f615-4150-b986-06ee5feaec44 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:54.155408 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15fc94-f615-4150-b986-06ee5feaec44 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:54.170477 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e15fc94-f615-4150-b986-06ee5feaec44 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:54.174025 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e15fc94-f615-4150-b986-06ee5feaec44 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:54.191935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25036780-1f53-4b92-8601-d134d284e50f None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:54.194226 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-7e15fc94-f615-4150-b986-06ee5feaec44 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 17 16:40:54.194226 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:54.194226 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:54.194226 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:54.194226 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:54.194226 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:54.194226 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:54.194226 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:54.194226 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:54.194226 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:54.194226 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:54.194226 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:54.194226 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:40:54.194226 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:40:54.194226 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:40:54.194226 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:40:54.196039 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:54.196039 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:54.196039 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:40:54.196039 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:40:54.196039 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:40:54.196039 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:40:54.196039 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:54.196039 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:54.196039 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:54.196039 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:54.196039 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:54.196039 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:54.196039 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:54.196039 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:54.196039 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:54.196039 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:54.196039 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:54.196039 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:54.198661 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:54.198661 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:54.198661 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:40:54.198661 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:40:54.198661 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:40:54.198661 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:40:54.198661 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:40:54.198661 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:40:54.198661 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 17 16:40:54.198661 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:54.198661 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 325/658] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 17 16:40:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:54.198661 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2119b95-c486-48cf-90c5-712cdb3a7de8 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:54.198661 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2119b95-c486-48cf-90c5-712cdb3a7de8 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:54.199875 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2119b95-c486-48cf-90c5-712cdb3a7de8 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:54.199875 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 326/659] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:54.199875 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2119b95-c486-48cf-90c5-712cdb3a7de8 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:54.217342 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:54.263858 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c24a1-f41e-4a88-a475-47158d1861cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:54.265025 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c24a1-f41e-4a88-a475-47158d1861cb None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:54.266052 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c24a1-f41e-4a88-a475-47158d1861cb None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:54.266519 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c24a1-f41e-4a88-a475-47158d1861cb None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:54.267766 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92c24a1-f41e-4a88-a475-47158d1861cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:54.268314 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92c24a1-f41e-4a88-a475-47158d1861cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:54.270855 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92c24a1-f41e-4a88-a475-47158d1861cb None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:54.294444 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 334/660] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:54.297117 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c24a1-f41e-4a88-a475-47158d1861cb None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:54.385521 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:54.389953 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2119b95-c486-48cf-90c5-712cdb3a7de8 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:54.428629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82824665-7b72-4398-8643-0cbe632fe576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:54.429397 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82824665-7b72-4398-8643-0cbe632fe576 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:54.429688 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82824665-7b72-4398-8643-0cbe632fe576 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:54.430105 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82824665-7b72-4398-8643-0cbe632fe576 None admin] PATH_INFO: `/v3/projects` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:54.431073 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82824665-7b72-4398-8643-0cbe632fe576 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:54.431393 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82824665-7b72-4398-8643-0cbe632fe576 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:54.432930 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82824665-7b72-4398-8643-0cbe632fe576 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:54.452455 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 327/661] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:40:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:54.453826 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec558a-a254-40bd-bc3a-359cb2f01827 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:54.454211 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec558a-a254-40bd-bc3a-359cb2f01827 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:54.454467 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec558a-a254-40bd-bc3a-359cb2f01827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:54.472005 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-e2ec558a-a254-40bd-bc3a-359cb2f01827 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:54.473295 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-e2ec558a-a254-40bd-bc3a-359cb2f01827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:54.480073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c24a1-f41e-4a88-a475-47158d1861cb None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:54.521271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb1cc7f-5f83-4b79-82ae-2715dfb98edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:54.521961 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1cc7f-5f83-4b79-82ae-2715dfb98edb None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:54.522249 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1cc7f-5f83-4b79-82ae-2715dfb98edb None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:54.522524 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1cc7f-5f83-4b79-82ae-2715dfb98edb None admin] PATH_INFO: `/v3/users` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:54.523389 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb1cc7f-5f83-4b79-82ae-2715dfb98edb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:54.523706 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb1cc7f-5f83-4b79-82ae-2715dfb98edb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:54.525185 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb1cc7f-5f83-4b79-82ae-2715dfb98edb None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:54.543843 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:54.545402 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 335/662] 10.222.0.40 () {56 vars in 1139 bytes} [Sun Jul 17 16:40:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:54.546297 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb1cc7f-5f83-4b79-82ae-2715dfb98edb None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:54.558384 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 328/663] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:54.559552 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec558a-a254-40bd-bc3a-359cb2f01827 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:54.694904 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec558a-a254-40bd-bc3a-359cb2f01827 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:54.714162 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:54.714162 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb1cc7f-5f83-4b79-82ae-2715dfb98edb None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:54.753342 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d4c867-5454-4f0d-8e7e-5305f22440e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:54.754166 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4c867-5454-4f0d-8e7e-5305f22440e1 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:54.754454 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4c867-5454-4f0d-8e7e-5305f22440e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:54.755031 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4c867-5454-4f0d-8e7e-5305f22440e1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:54.764257 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d4c867-5454-4f0d-8e7e-5305f22440e1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:54.766349 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d4c867-5454-4f0d-8e7e-5305f22440e1 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:54.773358 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785bb09-1123-41b0-ac63-2d5d2c956f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:54.774516 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785bb09-1123-41b0-ac63-2d5d2c956f2a None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:54.774977 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785bb09-1123-41b0-ac63-2d5d2c956f2a None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:54.775428 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3785bb09-1123-41b0-ac63-2d5d2c956f2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:54.776939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785bb09-1123-41b0-ac63-2d5d2c956f2a None admin] RBAC: Adding query filter params (role.id=adb9c27c6e5246fdb0cf386358a822cd, scope.project.id=927057f9a5af4833aeac9992203a223c, user.id=9e13bcc0f657445cbbdea5b78563f0d3) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:54.777458 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785bb09-1123-41b0-ac63-2d5d2c956f2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:54.779980 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3785bb09-1123-41b0-ac63-2d5d2c956f2a None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:54.797313 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 329/664] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:40:54 2022] GET /identity/v3/role_assignments?user.id=9e13bcc0f657445cbbdea5b78563f0d3&scope.project.id=927057f9a5af4833aeac9992203a223c&role.id=adb9c27c6e5246fdb0cf386358a822cd => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:54.797313 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785bb09-1123-41b0-ac63-2d5d2c956f2a None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:54.797690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-c1d4c867-5454-4f0d-8e7e-5305f22440e1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 17 16:40:54.797690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:54.797690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:54.797690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:54.797690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:54.797690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:54.797690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:54.797690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:54.797690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:54.797690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:54.797690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:54.797690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:54.797690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:40:54.797690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:40:54.797690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:40:54.797690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:40:54.799378 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:54.799378 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:54.799378 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:40:54.799378 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:40:54.799378 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:40:54.799378 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:40:54.799378 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:54.799378 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:54.799378 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:54.799378 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:54.799378 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:54.799378 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:54.799378 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:54.799378 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:54.799378 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:54.799378 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:54.799378 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:54.799378 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:54.810650 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:54.810650 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:54.810650 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:40:54.810650 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:40:54.810650 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:40:54.810650 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:40:54.810650 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:40:54.810650 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:40:54.810650 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 17 16:40:54.810650 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:54.810650 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 336/665] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:40:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:54.810650 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d4c867-5454-4f0d-8e7e-5305f22440e1 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:54.922913 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:55.010890 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d4c867-5454-4f0d-8e7e-5305f22440e1 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:55.011141 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3785bb09-1123-41b0-ac63-2d5d2c956f2a None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:55.078701 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4357f281-c8a9-4207-9676-b83a7bec62dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:55.079576 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357f281-c8a9-4207-9676-b83a7bec62dd None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:55.080035 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357f281-c8a9-4207-9676-b83a7bec62dd None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:55.080447 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357f281-c8a9-4207-9676-b83a7bec62dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:55.092518 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3dae7-e606-4654-84c9-bd27a631da16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:55.094546 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3dae7-e606-4654-84c9-bd27a631da16 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:55.095040 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3dae7-e606-4654-84c9-bd27a631da16 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:55.095540 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3dae7-e606-4654-84c9-bd27a631da16 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:55.119660 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb3dae7-e606-4654-84c9-bd27a631da16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:55.135073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-7eb3dae7-e606-4654-84c9-bd27a631da16 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:40:55.135073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:55.135073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:55.135073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:55.135073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:55.135073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:55.135073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:55.135073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:55.135073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:55.135073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:55.135073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:55.135073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:55.135073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:55.135073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:55.135073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:55.135073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:55.137017 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:55.137017 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:55.137017 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:55.137017 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:55.137017 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:55.137017 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:55.137017 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:55.137017 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:55.137017 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:55.137017 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:55.137017 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:55.137017 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:55.137017 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:55.137017 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:55.137017 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:55.137017 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:55.137978 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:55.137978 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:55.137978 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:55.137978 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:55.137978 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:55.137978 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:55.137978 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:55.137978 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:55.137978 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:55.137978 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:55.137978 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:40:55.137978 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:55.145159 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 337/666] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 17 16:40:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:55.147694 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3dae7-e606-4654-84c9-bd27a631da16 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:55.200005 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:55.237852 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4357f281-c8a9-4207-9676-b83a7bec62dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:55.240756 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4357f281-c8a9-4207-9676-b83a7bec62dd None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:55.363164 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:55.364926 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3dae7-e606-4654-84c9-bd27a631da16 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:55.448730 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c14390-3fe1-449c-8187-6f3accc3ca88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:55.449935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c14390-3fe1-449c-8187-6f3accc3ca88 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:55.450442 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c14390-3fe1-449c-8187-6f3accc3ca88 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:55.450930 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c14390-3fe1-449c-8187-6f3accc3ca88 None admin] PATH_INFO: `/v3/users` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:55.452573 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c14390-3fe1-449c-8187-6f3accc3ca88 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:55.453192 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c14390-3fe1-449c-8187-6f3accc3ca88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:55.453804 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 330/667] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:40:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 661 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 17 16:40:55.455611 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4357f281-c8a9-4207-9676-b83a7bec62dd None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:55.455936 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c14390-3fe1-449c-8187-6f3accc3ca88 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:55.504126 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 338/668] 10.222.0.40 () {56 vars in 1137 bytes} [Sun Jul 17 16:40:55 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:55.506058 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c14390-3fe1-449c-8187-6f3accc3ca88 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:55.691304 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:55.693040 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4357f281-c8a9-4207-9676-b83a7bec62dd None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:55.742578 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c14390-3fe1-449c-8187-6f3accc3ca88 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:55.781604 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c266a-4622-42d0-b312-84fba599bb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:55.782977 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c266a-4622-42d0-b312-84fba599bb8d None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:55.783590 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c266a-4622-42d0-b312-84fba599bb8d None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:55.784168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c266a-4622-42d0-b312-84fba599bb8d None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:55.786027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c266a-4622-42d0-b312-84fba599bb8d None admin] RBAC: Adding query filter params (name=service) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:55.786733 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c266a-4622-42d0-b312-84fba599bb8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:55.789629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c266a-4622-42d0-b312-84fba599bb8d None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:55.813416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 331/669] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 17 16:40:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:55.815319 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c266a-4622-42d0-b312-84fba599bb8d None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:55.839449 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a868f-432d-42ce-ab45-b1e9d6cbdfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:55.841368 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a868f-432d-42ce-ab45-b1e9d6cbdfec None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:55.842019 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a868f-432d-42ce-ab45-b1e9d6cbdfec None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:55.842616 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a868f-432d-42ce-ab45-b1e9d6cbdfec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:55.866566 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797a868f-432d-42ce-ab45-b1e9d6cbdfec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:55.869925 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797a868f-432d-42ce-ab45-b1e9d6cbdfec None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:55.874053 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:55.910704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-797a868f-432d-42ce-ab45-b1e9d6cbdfec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:55.910704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:55.910704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:55.910704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:55.910704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:55.910704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:55.910704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:55.910704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:55.910704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:55.910704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:55.910704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:55.910704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:55.910704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:55.910704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:55.910704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:55.912464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:55.912464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:55.912464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:55.912464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:55.912464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:55.912464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:55.912464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:55.912464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:55.912464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:55.912464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:55.912464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:55.912464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:55.912464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:55.912464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:55.912464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:55.912464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:55.912464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:55.914285 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:55.914285 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:55.914285 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:55.914285 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:55.914285 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 339/670] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 409 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:55.916285 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a868f-432d-42ce-ab45-b1e9d6cbdfec None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:56.008399 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c266a-4622-42d0-b312-84fba599bb8d None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:56.051270 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b1aba3-c9d0-4144-b423-86d5f6d8d041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:56.051270 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1aba3-c9d0-4144-b423-86d5f6d8d041 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:56.051664 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1aba3-c9d0-4144-b423-86d5f6d8d041 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:56.051963 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b1aba3-c9d0-4144-b423-86d5f6d8d041 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:56.102317 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a868f-432d-42ce-ab45-b1e9d6cbdfec None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:56.138550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b1aba3-c9d0-4144-b423-86d5f6d8d041 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:56.140182 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b1aba3-c9d0-4144-b423-86d5f6d8d041 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:56.159126 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:56.171677 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:56.183154 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b940053-ce6d-427e-ae7b-e93722d50983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:56.183782 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b940053-ce6d-427e-ae7b-e93722d50983 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:56.184124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b940053-ce6d-427e-ae7b-e93722d50983 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:56.184358 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b940053-ce6d-427e-ae7b-e93722d50983 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:56.199361 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b940053-ce6d-427e-ae7b-e93722d50983 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:56.200245 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b940053-ce6d-427e-ae7b-e93722d50983 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:56.223343 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-8b940053-ce6d-427e-ae7b-e93722d50983 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:56.223343 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:56.223343 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:56.223343 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:56.223343 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:56.223343 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:56.223343 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:56.223343 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:56.223343 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:56.223343 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:56.223343 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:56.223343 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:56.223343 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:56.223343 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:56.223343 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:56.225474 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:56.225474 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:56.225474 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:56.225474 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:56.225474 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:56.225474 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:56.225474 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:56.225474 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:56.225474 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:56.225474 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:56.225474 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:56.225474 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:56.225474 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:56.225474 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:56.225474 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:56.225474 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:56.225474 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:56.227369 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:56.227369 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:56.227369 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:56.227369 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:56.227369 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 340/671] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:56.227369 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b940053-ce6d-427e-ae7b-e93722d50983 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:56.297510 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 332/672] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:40:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 17 16:40:56.299651 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397773c-8e7a-437c-b01a-07a7b41febc9 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:56.299999 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397773c-8e7a-437c-b01a-07a7b41febc9 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:56.300346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397773c-8e7a-437c-b01a-07a7b41febc9 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:56.302124 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 333/673] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:56.306671 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397773c-8e7a-437c-b01a-07a7b41febc9 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:56.378718 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b940053-ce6d-427e-ae7b-e93722d50983 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:56.431888 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f225fd-9998-45d9-aeec-8457fc3be058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:56.432770 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f225fd-9998-45d9-aeec-8457fc3be058 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:56.433278 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f225fd-9998-45d9-aeec-8457fc3be058 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:56.433739 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f225fd-9998-45d9-aeec-8457fc3be058 None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:56.434964 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f225fd-9998-45d9-aeec-8457fc3be058 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:56.435483 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f225fd-9998-45d9-aeec-8457fc3be058 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:56.439239 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f225fd-9998-45d9-aeec-8457fc3be058 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:56.458567 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397773c-8e7a-437c-b01a-07a7b41febc9 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:56.461368 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 341/674] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:56.462722 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f225fd-9998-45d9-aeec-8457fc3be058 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:56.490895 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:56.504126 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa9f3fc-86a9-47af-8170-8a9571daede2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:56.505759 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9f3fc-86a9-47af-8170-8a9571daede2 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:56.505759 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9f3fc-86a9-47af-8170-8a9571daede2 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:56.505759 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9f3fc-86a9-47af-8170-8a9571daede2 None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:56.506431 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa9f3fc-86a9-47af-8170-8a9571daede2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:56.506747 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa9f3fc-86a9-47af-8170-8a9571daede2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:56.508163 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa9f3fc-86a9-47af-8170-8a9571daede2 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:56.525252 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 334/675] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:56.528077 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a64e4f-0482-400d-a5ec-f881f59c2758 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:56.528547 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a64e4f-0482-400d-a5ec-f881f59c2758 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:56.529056 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a64e4f-0482-400d-a5ec-f881f59c2758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:56.538808 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:56.562002 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-56a64e4f-0482-400d-a5ec-f881f59c2758 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:56.564019 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-56a64e4f-0482-400d-a5ec-f881f59c2758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:56.623789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f225fd-9998-45d9-aeec-8457fc3be058 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:56.673113 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6d26f-5ca2-4823-a5f9-e8c188f9fd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:56.674420 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6d26f-5ca2-4823-a5f9-e8c188f9fd02 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:56.685064 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6d26f-5ca2-4823-a5f9-e8c188f9fd02 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:56.685332 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 335/676] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:56.685492 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6d26f-5ca2-4823-a5f9-e8c188f9fd02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:56.685638 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a64e4f-0482-400d-a5ec-f881f59c2758 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:56.711340 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f6d26f-5ca2-4823-a5f9-e8c188f9fd02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:56.714685 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f6d26f-5ca2-4823-a5f9-e8c188f9fd02 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:56.755094 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-41f6d26f-5ca2-4823-a5f9-e8c188f9fd02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:56.755094 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:56.755094 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:56.755094 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:56.755094 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:56.755094 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:56.755094 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:56.755094 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:56.755094 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:56.755094 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:56.755094 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:56.755094 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:56.755094 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:56.755094 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:56.755094 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:56.756057 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:56.756057 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:56.756057 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:56.756057 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:56.756057 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:56.756057 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:56.756057 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:56.756057 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:56.756057 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:56.756057 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:56.756057 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:56.756057 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:56.756057 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:56.756057 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:56.756057 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:56.756057 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:56.756057 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:56.756969 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:56.756969 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:56.756969 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:56.756969 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:56.757188 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 342/677] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:56.758289 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6d26f-5ca2-4823-a5f9-e8c188f9fd02 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:56.823413 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a64e4f-0482-400d-a5ec-f881f59c2758 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:56.848671 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:56.886871 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37560f2-1ffb-4ed5-bf5c-d7d3cf38ef31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:56.888603 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37560f2-1ffb-4ed5-bf5c-d7d3cf38ef31 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:56.889154 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37560f2-1ffb-4ed5-bf5c-d7d3cf38ef31 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:56.889652 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37560f2-1ffb-4ed5-bf5c-d7d3cf38ef31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:56.900188 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:56.909293 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6d26f-5ca2-4823-a5f9-e8c188f9fd02 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:56.909906 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37560f2-1ffb-4ed5-bf5c-d7d3cf38ef31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:56.912673 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37560f2-1ffb-4ed5-bf5c-d7d3cf38ef31 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:56.939021 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-c37560f2-1ffb-4ed5-bf5c-d7d3cf38ef31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:40:56.939021 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:56.939021 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:56.939021 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:56.939021 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:56.939021 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:56.939021 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:56.939021 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:56.939021 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:56.939021 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:56.939021 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:56.939021 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:56.939021 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:40:56.939021 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:40:56.939021 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:40:56.939983 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:40:56.939983 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:56.939983 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:56.939983 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:56.939983 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:56.939983 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:56.939983 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:56.939983 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:56.939983 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:56.939983 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:56.939983 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:56.939983 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:56.939983 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:56.939983 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:56.939983 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:56.939983 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:56.939983 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:56.940908 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:40:56.940908 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:40:56.940908 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:40:56.940908 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:40:56.940908 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:40:56.940908 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:40:56.940908 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:40:56.940908 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:56.949735 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 336/678] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:40:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:56.950679 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37560f2-1ffb-4ed5-bf5c-d7d3cf38ef31 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:56.975562 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad9a6e-6184-4532-ae16-aaed4a13687e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:56.976985 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad9a6e-6184-4532-ae16-aaed4a13687e None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:56.977488 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad9a6e-6184-4532-ae16-aaed4a13687e None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:56.977999 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad9a6e-6184-4532-ae16-aaed4a13687e None admin] PATH_INFO: `/v3/users/swift` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:56.992424 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbad9a6e-6184-4532-ae16-aaed4a13687e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:56.995221 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbad9a6e-6184-4532-ae16-aaed4a13687e None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:57.024220 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-cbad9a6e-6184-4532-ae16-aaed4a13687e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 17 16:40:57.024220 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:57.024220 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:57.024220 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:57.024220 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:57.024220 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:57.024220 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:57.024220 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:57.024220 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:57.024220 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:57.024220 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:57.024220 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:57.024220 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:40:57.024220 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:40:57.024220 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:40:57.024220 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:40:57.025333 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:57.025333 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:57.025333 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:40:57.025333 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:40:57.025333 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:40:57.025333 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:40:57.025333 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:57.025333 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:57.025333 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:57.025333 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:57.025333 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:57.025333 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:57.025333 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:57.025333 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:57.025333 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:57.025333 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:57.025333 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:57.025333 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:57.026271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:57.026271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:57.026271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:40:57.026271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:40:57.026271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:40:57.026271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:40:57.026271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:40:57.026271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:40:57.026271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 17 16:40:57.026271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:57.026808 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 343/679] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:40:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:57.027963 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad9a6e-6184-4532-ae16-aaed4a13687e None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:57.135674 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37560f2-1ffb-4ed5-bf5c-d7d3cf38ef31 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:57.193448 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a77b4c0-6158-4810-bbbf-693b10402309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:57.194966 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77b4c0-6158-4810-bbbf-693b10402309 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:57.195371 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77b4c0-6158-4810-bbbf-693b10402309 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:57.195783 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77b4c0-6158-4810-bbbf-693b10402309 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:57.210945 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a77b4c0-6158-4810-bbbf-693b10402309 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:40:57.213081 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad9a6e-6184-4532-ae16-aaed4a13687e None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:57.220842 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-1a77b4c0-6158-4810-bbbf-693b10402309 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:40:57.220842 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:57.220842 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:57.220842 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:57.220842 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:57.220842 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:57.220842 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:57.220842 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:57.220842 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:57.220842 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:57.220842 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:57.220842 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:57.220842 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:40:57.220842 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:40:57.220842 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:40:57.220842 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:40:57.222633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:40:57.222633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:40:57.222633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:40:57.222633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:40:57.222633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:57.222633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:57.222633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:57.222633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:57.222633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:57.222633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:57.222633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:57.222633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:57.222633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:57.222633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:57.222633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:57.222633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:57.229999 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:57.229999 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:57.229999 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:57.229999 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:57.229999 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:40:57.229999 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:40:57.229999 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:40:57.229999 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:40:57.229999 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:40:57.229999 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:40:57.229999 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:40:57.229999 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:57.229999 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 337/680] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:40:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:57.229999 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a77b4c0-6158-4810-bbbf-693b10402309 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:57.270477 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7086da-5f47-4493-b635-474598df4ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:57.271543 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7086da-5f47-4493-b635-474598df4ab4 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:57.272054 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7086da-5f47-4493-b635-474598df4ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:57.272534 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7086da-5f47-4493-b635-474598df4ab4 None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:57.273931 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7086da-5f47-4493-b635-474598df4ab4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:57.274489 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7086da-5f47-4493-b635-474598df4ab4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:57.276993 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7086da-5f47-4493-b635-474598df4ab4 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:57.287782 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:57.306512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 344/681] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:57.306512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7086da-5f47-4493-b635-474598df4ab4 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:57.335143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:57.410405 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a77b4c0-6158-4810-bbbf-693b10402309 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:57.442468 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7086da-5f47-4493-b635-474598df4ab4 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:57.456311 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2de3ac-2b30-48cf-a213-d52343ecb0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:57.457156 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2de3ac-2b30-48cf-a213-d52343ecb0eb None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:57.457523 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2de3ac-2b30-48cf-a213-d52343ecb0eb None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:57.457975 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2de3ac-2b30-48cf-a213-d52343ecb0eb None admin] PATH_INFO: `/v3/projects` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:57.459028 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2de3ac-2b30-48cf-a213-d52343ecb0eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:57.459413 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2de3ac-2b30-48cf-a213-d52343ecb0eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:57.461088 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2de3ac-2b30-48cf-a213-d52343ecb0eb None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:57.478376 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 338/682] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:40:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:57.479764 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2de3ac-2b30-48cf-a213-d52343ecb0eb None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:57.487032 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21efea7f-9a06-47fa-8fc8-6fce1ecd9a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:57.487958 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21efea7f-9a06-47fa-8fc8-6fce1ecd9a48 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:57.488408 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21efea7f-9a06-47fa-8fc8-6fce1ecd9a48 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:57.488897 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21efea7f-9a06-47fa-8fc8-6fce1ecd9a48 None admin] PATH_INFO: `/v3/users` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:57.490343 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21efea7f-9a06-47fa-8fc8-6fce1ecd9a48 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:57.490840 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21efea7f-9a06-47fa-8fc8-6fce1ecd9a48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:57.493370 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21efea7f-9a06-47fa-8fc8-6fce1ecd9a48 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:57.529295 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 345/683] 10.222.0.40 () {56 vars in 1135 bytes} [Sun Jul 17 16:40:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:57.529295 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21efea7f-9a06-47fa-8fc8-6fce1ecd9a48 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:57.627396 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:57.639678 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2de3ac-2b30-48cf-a213-d52343ecb0eb None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:57.646725 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:57.696110 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e7135-9f33-484f-b900-eee6cb99302f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:57.696904 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e7135-9f33-484f-b900-eee6cb99302f None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:57.697295 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e7135-9f33-484f-b900-eee6cb99302f None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:57.697709 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e7135-9f33-484f-b900-eee6cb99302f None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:57.698392 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21efea7f-9a06-47fa-8fc8-6fce1ecd9a48 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:57.698738 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e7135-9f33-484f-b900-eee6cb99302f None admin] RBAC: Adding query filter params (name=admin) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:57.699229 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e7135-9f33-484f-b900-eee6cb99302f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:57.700823 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e7135-9f33-484f-b900-eee6cb99302f None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:57.711697 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 339/684] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:40:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:57.713077 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a02a0-253a-4f4c-b3a0-1b4385508c09 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:57.713440 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a02a0-253a-4f4c-b3a0-1b4385508c09 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:57.713732 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a02a0-253a-4f4c-b3a0-1b4385508c09 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:57.715057 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 340/685] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:40:57.715805 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a02a0-253a-4f4c-b3a0-1b4385508c09 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:57.749148 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4859564b-1575-4379-b90d-c386f120717e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:57.750962 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859564b-1575-4379-b90d-c386f120717e None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:57.751432 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859564b-1575-4379-b90d-c386f120717e None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:57.751914 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859564b-1575-4379-b90d-c386f120717e None admin] PATH_INFO: `/v3/projects/service` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:57.764023 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4859564b-1575-4379-b90d-c386f120717e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:57.767100 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4859564b-1575-4379-b90d-c386f120717e None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:57.792026 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-4859564b-1575-4379-b90d-c386f120717e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:40:57.792026 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:57.792026 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:57.792026 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:57.792026 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:57.792026 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:57.792026 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:57.792026 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:57.792026 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:57.792026 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:57.792026 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:57.792026 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:57.792026 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:40:57.792026 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:40:57.792026 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:40:57.793379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:40:57.793379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:57.793379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:57.793379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:57.793379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:57.793379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:57.793379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:57.793379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:57.793379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:57.793379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:57.793379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:57.793379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:57.793379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:57.793379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:57.793379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:57.793379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:57.793379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:57.794486 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:40:57.794486 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:40:57.794486 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:40:57.794486 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:40:57.794486 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:40:57.794486 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:40:57.794486 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:40:57.794486 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:57.794486 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 346/686] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:40:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:57.794996 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4859564b-1575-4379-b90d-c386f120717e None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:57.860466 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a02a0-253a-4f4c-b3a0-1b4385508c09 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:57.900980 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be2c330-ed78-422d-a5dc-315092fc5600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:57.901781 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2c330-ed78-422d-a5dc-315092fc5600 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:57.904991 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2c330-ed78-422d-a5dc-315092fc5600 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:57.904991 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be2c330-ed78-422d-a5dc-315092fc5600 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:57.904991 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be2c330-ed78-422d-a5dc-315092fc5600 None admin] RBAC: Adding query filter params (role.id=adb9c27c6e5246fdb0cf386358a822cd, scope.project.id=927057f9a5af4833aeac9992203a223c, user.id=9663edc8502b4aecb5317b24e4a5816b) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:57.904991 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be2c330-ed78-422d-a5dc-315092fc5600 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:57.905608 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be2c330-ed78-422d-a5dc-315092fc5600 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:57.912555 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 341/687] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:40:57 2022] GET /identity/v3/role_assignments?user.id=9663edc8502b4aecb5317b24e4a5816b&scope.project.id=927057f9a5af4833aeac9992203a223c&role.id=adb9c27c6e5246fdb0cf386358a822cd => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:57.913474 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be2c330-ed78-422d-a5dc-315092fc5600 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:57.930391 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4859564b-1575-4379-b90d-c386f120717e None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:57.947335 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:57.978388 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:57.987919 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe7fb8-d813-44b3-9532-1a597ac37fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:57.988613 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe7fb8-d813-44b3-9532-1a597ac37fd3 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:57.988955 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe7fb8-d813-44b3-9532-1a597ac37fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:57.989489 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe7fb8-d813-44b3-9532-1a597ac37fd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:58.072906 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be2c330-ed78-422d-a5dc-315092fc5600 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:58.099476 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfe7fb8-d813-44b3-9532-1a597ac37fd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:58.102980 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfe7fb8-d813-44b3-9532-1a597ac37fd3 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:58.118623 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74514ba6-be1c-4e86-9ecb-57c963277f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:58.119467 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74514ba6-be1c-4e86-9ecb-57c963277f36 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:58.119757 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74514ba6-be1c-4e86-9ecb-57c963277f36 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:58.120049 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74514ba6-be1c-4e86-9ecb-57c963277f36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:58.134122 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74514ba6-be1c-4e86-9ecb-57c963277f36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:58.135712 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74514ba6-be1c-4e86-9ecb-57c963277f36 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:58.156762 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-74514ba6-be1c-4e86-9ecb-57c963277f36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:58.156762 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:58.156762 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:58.156762 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:58.156762 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:58.156762 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:58.156762 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:58.156762 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:58.156762 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:58.156762 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:58.156762 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:58.156762 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:58.156762 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:58.156762 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:58.156762 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:58.158596 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:58.158596 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:58.158596 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:58.158596 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:58.158596 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:58.158596 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:58.158596 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:58.158596 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:58.158596 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:58.158596 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:58.158596 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:58.158596 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:58.158596 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:58.158596 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:58.158596 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:58.158596 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:58.158596 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:58.160399 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:58.160399 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:58.160399 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:58.160399 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:58.160399 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 342/688] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:58.160399 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c5f97-80da-411f-925e-d0094cb30d49 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:58.161112 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c5f97-80da-411f-925e-d0094cb30d49 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:58.161112 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c5f97-80da-411f-925e-d0094cb30d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:58.196848 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-809c5f97-80da-411f-925e-d0094cb30d49 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:40:58.198984 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-809c5f97-80da-411f-925e-d0094cb30d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:40:58.306447 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 347/689] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:40:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 17 16:40:58.307758 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe7fb8-d813-44b3-9532-1a597ac37fd3 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:58.321169 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:58.331419 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 343/690] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:40:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:40:58.333634 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c5f97-80da-411f-925e-d0094cb30d49 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:58.380420 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:58.498864 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe7fb8-d813-44b3-9532-1a597ac37fd3 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:58.528594 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c5f97-80da-411f-925e-d0094cb30d49 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:58.563165 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5122c-c4c5-43f1-9ae7-0098a78d6a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:58.564276 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5122c-c4c5-43f1-9ae7-0098a78d6a22 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:58.564777 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5122c-c4c5-43f1-9ae7-0098a78d6a22 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:58.565316 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5122c-c4c5-43f1-9ae7-0098a78d6a22 None admin] PATH_INFO: `/v3/projects` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:58.566845 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a5122c-c4c5-43f1-9ae7-0098a78d6a22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:58.567386 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a5122c-c4c5-43f1-9ae7-0098a78d6a22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:58.570444 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a5122c-c4c5-43f1-9ae7-0098a78d6a22 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:58.579441 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc4964-7b68-4e14-a55c-975af22dc3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:58.580136 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc4964-7b68-4e14-a55c-975af22dc3fa None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:58.580419 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc4964-7b68-4e14-a55c-975af22dc3fa None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:58.580701 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc4964-7b68-4e14-a55c-975af22dc3fa None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:58.581511 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcc4964-7b68-4e14-a55c-975af22dc3fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:58.581946 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcc4964-7b68-4e14-a55c-975af22dc3fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:58.583276 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcc4964-7b68-4e14-a55c-975af22dc3fa None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:58.594357 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 348/691] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:40:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:58.596247 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5122c-c4c5-43f1-9ae7-0098a78d6a22 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:58.604885 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 344/692] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:58.606100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc4964-7b68-4e14-a55c-975af22dc3fa None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:58.680646 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:58.735265 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc4964-7b68-4e14-a55c-975af22dc3fa None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:58.762323 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:58.776197 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd3268-eb5c-44e9-8408-470e51f4bc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:58.777054 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd3268-eb5c-44e9-8408-470e51f4bc30 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:58.777336 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd3268-eb5c-44e9-8408-470e51f4bc30 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:58.777610 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd3268-eb5c-44e9-8408-470e51f4bc30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:58.778750 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5122c-c4c5-43f1-9ae7-0098a78d6a22 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:58.791483 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd3268-eb5c-44e9-8408-470e51f4bc30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:58.793072 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd3268-eb5c-44e9-8408-470e51f4bc30 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:58.819886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-f9bd3268-eb5c-44e9-8408-470e51f4bc30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:58.819886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:58.819886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:58.819886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:58.819886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:58.819886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:58.819886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:58.819886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:58.819886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:58.819886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:58.819886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:58.819886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:58.819886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:58.819886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:58.819886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:58.821694 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:58.821694 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:58.821694 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:58.821694 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:58.821694 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:58.821694 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:58.821694 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:58.821694 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:58.821694 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:58.821694 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:58.821694 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:58.821694 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:58.821694 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:58.821694 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:58.821694 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:58.821694 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:58.821694 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:58.823413 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:58.823413 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:58.823413 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:58.823413 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:40:58.823413 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 345/693] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:58.823413 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd3268-eb5c-44e9-8408-470e51f4bc30 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:58.847893 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa2664-5d4e-45ef-8dd4-b6888d25b69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:58.849259 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa2664-5d4e-45ef-8dd4-b6888d25b69d None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:58.849684 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa2664-5d4e-45ef-8dd4-b6888d25b69d None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:58.850147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa2664-5d4e-45ef-8dd4-b6888d25b69d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:58.866691 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa2664-5d4e-45ef-8dd4-b6888d25b69d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:58.868617 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa2664-5d4e-45ef-8dd4-b6888d25b69d None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:58.892061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-85aa2664-5d4e-45ef-8dd4-b6888d25b69d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:58.892061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:58.892061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:58.892061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:58.892061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:58.892061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:58.892061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:58.892061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:58.892061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:58.892061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:58.892061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:58.892061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:58.892061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:40:58.892061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:40:58.892061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:40:58.893839 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:40:58.893839 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:58.893839 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:58.893839 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:58.893839 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:58.893839 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:58.893839 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:58.893839 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:58.893839 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:58.893839 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:58.893839 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:58.893839 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:58.893839 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:58.893839 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:58.893839 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:58.893839 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:58.893839 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:58.895521 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:40:58.895521 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:40:58.895521 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:40:58.895521 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:58.898144 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 349/694] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:40:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:58.899134 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa2664-5d4e-45ef-8dd4-b6888d25b69d None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:58.968553 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd3268-eb5c-44e9-8408-470e51f4bc30 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:58.989607 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:59.028482 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90837727-4b46-4060-903d-db4cf6c48655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:59.029907 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90837727-4b46-4060-903d-db4cf6c48655 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:59.030357 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90837727-4b46-4060-903d-db4cf6c48655 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:59.031446 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90837727-4b46-4060-903d-db4cf6c48655 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:59.034780 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90837727-4b46-4060-903d-db4cf6c48655 None admin] RBAC: Adding query filter params (role.id=adb9c27c6e5246fdb0cf386358a822cd, scope.project.id=927057f9a5af4833aeac9992203a223c, user.id=0b7122fc744d446aa60b25844c68f15b) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:59.035618 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90837727-4b46-4060-903d-db4cf6c48655 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:59.038965 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90837727-4b46-4060-903d-db4cf6c48655 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:59.047989 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa2664-5d4e-45ef-8dd4-b6888d25b69d None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:59.051696 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 346/695] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:40:58 2022] GET /identity/v3/role_assignments?user.id=0b7122fc744d446aa60b25844c68f15b&scope.project.id=927057f9a5af4833aeac9992203a223c&role.id=adb9c27c6e5246fdb0cf386358a822cd => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:59.052513 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90837727-4b46-4060-903d-db4cf6c48655 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:59.136815 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fee78-1f8e-418a-b0b0-69239c9ba9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:59.139007 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fee78-1f8e-418a-b0b0-69239c9ba9cc None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:59.139924 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fee78-1f8e-418a-b0b0-69239c9ba9cc None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:59.141188 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fee78-1f8e-418a-b0b0-69239c9ba9cc None admin] PATH_INFO: `/v3/users/nova` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:59.154043 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08fee78-1f8e-418a-b0b0-69239c9ba9cc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:59.157003 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08fee78-1f8e-418a-b0b0-69239c9ba9cc None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:59.181082 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-a08fee78-1f8e-418a-b0b0-69239c9ba9cc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 17 16:40:59.181082 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:40:59.181082 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:40:59.181082 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:40:59.181082 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:40:59.181082 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:40:59.181082 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:40:59.181082 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:40:59.181082 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:40:59.181082 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:40:59.181082 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:40:59.181082 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:40:59.181082 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:40:59.181082 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:40:59.181082 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:40:59.181082 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:40:59.182807 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:40:59.182807 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:40:59.182807 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:40:59.182807 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:40:59.182807 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:40:59.182807 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:40:59.182807 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:40:59.182807 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:40:59.182807 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:40:59.182807 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:40:59.182807 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:40:59.182807 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:40:59.182807 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:40:59.182807 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:40:59.182807 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:40:59.182807 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:40:59.182807 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:40:59.182807 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:40:59.184616 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:40:59.184616 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:40:59.184616 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:40:59.184616 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:40:59.184616 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:40:59.184616 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:40:59.184616 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:40:59.184616 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:40:59.184616 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 17 16:40:59.184616 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:40:59.190695 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 350/696] 10.222.0.40 () {56 vars in 1093 bytes} [Sun Jul 17 16:40:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:40:59.192825 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fee78-1f8e-418a-b0b0-69239c9ba9cc None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:59.200867 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:59.246994 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90837727-4b46-4060-903d-db4cf6c48655 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:59.333985 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2476bb-6111-4d04-9741-221b625cb5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:59.335668 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2476bb-6111-4d04-9741-221b625cb5c3 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:59.336367 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2476bb-6111-4d04-9741-221b625cb5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:59.336940 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2476bb-6111-4d04-9741-221b625cb5c3 None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:59.341428 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2476bb-6111-4d04-9741-221b625cb5c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:59.342031 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2476bb-6111-4d04-9741-221b625cb5c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:59.344840 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2476bb-6111-4d04-9741-221b625cb5c3 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:59.381171 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 347/697] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:59.382796 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2476bb-6111-4d04-9741-221b625cb5c3 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:59.418881 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fee78-1f8e-418a-b0b0-69239c9ba9cc None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:59.486734 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:59.504118 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db5359-5e85-49d4-ba70-53e40d3337fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:59.506567 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db5359-5e85-49d4-ba70-53e40d3337fb None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:59.507463 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db5359-5e85-49d4-ba70-53e40d3337fb None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:59.508264 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db5359-5e85-49d4-ba70-53e40d3337fb None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:59.510183 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0db5359-5e85-49d4-ba70-53e40d3337fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:59.511249 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0db5359-5e85-49d4-ba70-53e40d3337fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:59.514628 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0db5359-5e85-49d4-ba70-53e40d3337fb None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:59.549168 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 351/698] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:40:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:59.550805 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db5359-5e85-49d4-ba70-53e40d3337fb None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:59.636706 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2476bb-6111-4d04-9741-221b625cb5c3 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:59.665347 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:40:59.692587 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1381cab-dd96-4fb4-b82b-bbec578183f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:59.694023 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1381cab-dd96-4fb4-b82b-bbec578183f6 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:59.694679 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1381cab-dd96-4fb4-b82b-bbec578183f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:59.696332 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1381cab-dd96-4fb4-b82b-bbec578183f6 None admin] PATH_INFO: `/v3/users` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:59.698574 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1381cab-dd96-4fb4-b82b-bbec578183f6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:40:59.699137 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1381cab-dd96-4fb4-b82b-bbec578183f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:59.702224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1381cab-dd96-4fb4-b82b-bbec578183f6 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:59.725927 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db5359-5e85-49d4-ba70-53e40d3337fb None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:40:59.752423 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 348/699] 10.222.0.40 () {56 vars in 1133 bytes} [Sun Jul 17 16:40:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:40:59.754135 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1381cab-dd96-4fb4-b82b-bbec578183f6 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:59.770098 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5eb25-4b71-4337-b2ff-e13189395b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:40:59.771157 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5eb25-4b71-4337-b2ff-e13189395b52 None admin] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:40:59.771686 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5eb25-4b71-4337-b2ff-e13189395b52 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:40:59.772162 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5eb25-4b71-4337-b2ff-e13189395b52 None admin] PATH_INFO: `/v3/users` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:40:59.773859 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab5eb25-4b71-4337-b2ff-e13189395b52 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:40:59.776675 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab5eb25-4b71-4337-b2ff-e13189395b52 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:40:59.921069 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 352/700] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 17 16:40:59 2022] POST /identity/v3/users => generated 254 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:40:59.922567 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5eb25-4b71-4337-b2ff-e13189395b52 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:40:59.979073 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:00.006195 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1381cab-dd96-4fb4-b82b-bbec578183f6 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:00.144101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b6e34-b488-45c5-832a-f3f5331583b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:00.145584 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b6e34-b488-45c5-832a-f3f5331583b8 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:00.146074 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b6e34-b488-45c5-832a-f3f5331583b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:00.146549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264b6e34-b488-45c5-832a-f3f5331583b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:00.174890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264b6e34-b488-45c5-832a-f3f5331583b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:00.177518 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264b6e34-b488-45c5-832a-f3f5331583b8 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:00.206325 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5eb25-4b71-4337-b2ff-e13189395b52 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:00.223611 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-264b6e34-b488-45c5-832a-f3f5331583b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:41:00.223611 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:00.223611 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:00.223611 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:00.223611 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:00.223611 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:00.223611 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:00.223611 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:00.223611 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:00.223611 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:00.223611 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:00.223611 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:00.223611 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:41:00.223611 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:41:00.223611 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:41:00.224561 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:41:00.224561 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:00.224561 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:00.224561 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:00.224561 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:00.224561 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:00.224561 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:00.224561 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:00.224561 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:00.224561 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:00.224561 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:00.224561 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:00.224561 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:00.224561 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:00.224561 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:00.224561 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:00.224561 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:00.225497 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:41:00.225497 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:41:00.225497 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:41:00.225497 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:41:00.225497 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:41:00.225497 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:41:00.225497 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:41:00.225497 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:00.227956 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 349/701] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:40:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 474 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:00.235828 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b6e34-b488-45c5-832a-f3f5331583b8 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:00.243618 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:00.294062 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b2298-d0da-4b2c-902c-62c56a73b11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:00.295456 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b2298-d0da-4b2c-902c-62c56a73b11e None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:00.295973 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b2298-d0da-4b2c-902c-62c56a73b11e None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:00.297856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b2298-d0da-4b2c-902c-62c56a73b11e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:00.387179 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b2298-d0da-4b2c-902c-62c56a73b11e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:00.389077 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b2298-d0da-4b2c-902c-62c56a73b11e None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:00.439981 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264b6e34-b488-45c5-832a-f3f5331583b8 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:00.517315 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 353/702] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:40:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 17 16:41:00.524329 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426fd0b2-ae67-45a1-9cb8-27e9546ef554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:00.527089 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426fd0b2-ae67-45a1-9cb8-27e9546ef554 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:00.527089 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426fd0b2-ae67-45a1-9cb8-27e9546ef554 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:00.527089 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426fd0b2-ae67-45a1-9cb8-27e9546ef554 None admin] PATH_INFO: `/v3/projects` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:00.528061 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426fd0b2-ae67-45a1-9cb8-27e9546ef554 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:00.528599 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426fd0b2-ae67-45a1-9cb8-27e9546ef554 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:00.529080 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b2298-d0da-4b2c-902c-62c56a73b11e None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:00.531800 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426fd0b2-ae67-45a1-9cb8-27e9546ef554 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:00.556067 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:00.561598 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 350/703] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:41:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:00.570832 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426fd0b2-ae67-45a1-9cb8-27e9546ef554 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:00.642386 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:00.766672 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b2298-d0da-4b2c-902c-62c56a73b11e None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:00.835224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426fd0b2-ae67-45a1-9cb8-27e9546ef554 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:00.838738 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809cd15e-3e31-4cc0-b05c-0d2768a17c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:00.840091 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809cd15e-3e31-4cc0-b05c-0d2768a17c8c None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:00.840590 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809cd15e-3e31-4cc0-b05c-0d2768a17c8c None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:00.841140 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809cd15e-3e31-4cc0-b05c-0d2768a17c8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:00.862551 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809cd15e-3e31-4cc0-b05c-0d2768a17c8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:00.865490 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809cd15e-3e31-4cc0-b05c-0d2768a17c8c None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:00.900959 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-809cd15e-3e31-4cc0-b05c-0d2768a17c8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:00.900959 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:00.900959 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:00.900959 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:00.900959 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:00.900959 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:00.900959 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:00.900959 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:00.900959 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:00.900959 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:00.900959 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:00.900959 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:00.900959 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:41:00.900959 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:41:00.900959 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:41:00.902689 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:41:00.902689 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:00.902689 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:00.902689 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:00.902689 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:00.902689 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:00.902689 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:00.902689 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:00.902689 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:00.902689 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:00.902689 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:00.902689 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:00.902689 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:00.902689 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:00.902689 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:00.902689 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:00.902689 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:00.905079 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:41:00.905079 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:41:00.905079 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:00.905079 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:00.905079 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 354/704] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:00.906322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f155b5-9c33-4245-a2f7-3e36e689201b None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:00.906786 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f155b5-9c33-4245-a2f7-3e36e689201b None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:00.907502 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f155b5-9c33-4245-a2f7-3e36e689201b None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:00.914087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 355/705] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:00.914087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f155b5-9c33-4245-a2f7-3e36e689201b None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:00.924905 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5d691-1527-4614-9b0e-5038df34759c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:00.925601 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5d691-1527-4614-9b0e-5038df34759c None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:00.925889 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5d691-1527-4614-9b0e-5038df34759c None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:00.926528 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5d691-1527-4614-9b0e-5038df34759c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:00.927494 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d5d691-1527-4614-9b0e-5038df34759c None admin] RBAC: Adding query filter params (role.id=adb9c27c6e5246fdb0cf386358a822cd, scope.project.id=927057f9a5af4833aeac9992203a223c, user.id=69482eeb1e8547aabca9f036b5c34f13) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:00.927917 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d5d691-1527-4614-9b0e-5038df34759c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:00.929678 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d5d691-1527-4614-9b0e-5038df34759c None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:00.941254 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 351/706] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:41:00 2022] GET /identity/v3/role_assignments?user.id=69482eeb1e8547aabca9f036b5c34f13&scope.project.id=927057f9a5af4833aeac9992203a223c&role.id=adb9c27c6e5246fdb0cf386358a822cd => generated 608 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:00.942996 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8894db1-7c40-4723-adcf-b35d6971a88c None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:00.943400 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8894db1-7c40-4723-adcf-b35d6971a88c None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:00.944039 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8894db1-7c40-4723-adcf-b35d6971a88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:00.982630 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-e8894db1-7c40-4723-adcf-b35d6971a88c None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:00.985891 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-e8894db1-7c40-4723-adcf-b35d6971a88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:01.057278 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:01.120017 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f155b5-9c33-4245-a2f7-3e36e689201b None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:01.188844 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 352/707] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:41:01.195633 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39ca8d-e90f-4504-b3a0-0645a09d7267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:01.197216 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39ca8d-e90f-4504-b3a0-0645a09d7267 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:01.197761 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39ca8d-e90f-4504-b3a0-0645a09d7267 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:01.198386 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b39ca8d-e90f-4504-b3a0-0645a09d7267 None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:01.200015 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b39ca8d-e90f-4504-b3a0-0645a09d7267 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:01.200660 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b39ca8d-e90f-4504-b3a0-0645a09d7267 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:01.204505 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b39ca8d-e90f-4504-b3a0-0645a09d7267 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:01.213733 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8894db1-7c40-4723-adcf-b35d6971a88c None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:01.228068 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:01.236682 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 356/708] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:41:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:01.243988 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39ca8d-e90f-4504-b3a0-0645a09d7267 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:01.355640 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8894db1-7c40-4723-adcf-b35d6971a88c None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:01.386047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b39ca8d-e90f-4504-b3a0-0645a09d7267 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:01.396985 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6814b-ad0a-4da3-ba94-e647a2b6fcc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:01.397959 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6814b-ad0a-4da3-ba94-e647a2b6fcc8 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:01.398560 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6814b-ad0a-4da3-ba94-e647a2b6fcc8 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:01.398829 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6814b-ad0a-4da3-ba94-e647a2b6fcc8 None admin] PATH_INFO: `/v3/services/network` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:01.399613 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6814b-ad0a-4da3-ba94-e647a2b6fcc8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:01.401140 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6814b-ad0a-4da3-ba94-e647a2b6fcc8 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:01.417383 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-8bf6814b-ad0a-4da3-ba94-e647a2b6fcc8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 17 16:41:01.417383 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:01.417383 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 17 16:41:01.417383 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 17 16:41:01.417383 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 17 16:41:01.417383 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 17 16:41:01.417383 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 17 16:41:01.417383 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:01.417383 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 17 16:41:01.417383 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:01.417383 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 17 16:41:01.417383 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:01.417383 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:01.417383 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:01.417383 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:01.417383 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:01.417383 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:01.418729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:01.418729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:01.418729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:01.418729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:01.418729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:01.418729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:01.418729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 17 16:41:01.418729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 17 16:41:01.418729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 17 16:41:01.418729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 17 16:41:01.418729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:01.418729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:01.418729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:01.418729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:01.418729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:01.418729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:01.419615 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:01.419615 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:01.419615 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:01.419615 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:01.419615 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:01.419615 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:01.419615 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:01.419615 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:01.419615 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:01.419615 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:01.419615 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 17 16:41:01.419615 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:01.419615 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 17 16:41:01.419615 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:01.420402 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 353/709] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:41:01 2022] GET /identity/v3/services/network => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:41:01.425619 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:01.428617 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6814b-ad0a-4da3-ba94-e647a2b6fcc8 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:01.434996 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51757852-fd1e-4706-9533-583dc9fd2a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:01.435630 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51757852-fd1e-4706-9533-583dc9fd2a10 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:01.436128 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51757852-fd1e-4706-9533-583dc9fd2a10 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:01.436619 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51757852-fd1e-4706-9533-583dc9fd2a10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:01.453061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51757852-fd1e-4706-9533-583dc9fd2a10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:01.455697 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51757852-fd1e-4706-9533-583dc9fd2a10 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:01.489132 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-51757852-fd1e-4706-9533-583dc9fd2a10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:41:01.489132 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:01.489132 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:01.489132 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:01.489132 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:01.489132 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:01.489132 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:01.489132 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:01.489132 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:01.489132 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:01.489132 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:01.489132 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:01.489132 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:41:01.489132 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:41:01.489132 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:41:01.490178 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:41:01.490178 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:01.490178 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:01.490178 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:01.490178 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:01.490178 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:01.490178 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:01.490178 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:01.490178 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:01.490178 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:01.490178 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:01.490178 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:01.490178 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:01.490178 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:01.490178 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:01.490178 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:01.490178 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:01.491109 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:41:01.491109 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:41:01.491109 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:41:01.491109 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:41:01.491109 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:41:01.491109 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:41:01.491109 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:41:01.491109 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:01.496946 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 357/710] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:41:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:41:01.503199 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51757852-fd1e-4706-9533-583dc9fd2a10 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:01.524597 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:01.563294 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6814b-ad0a-4da3-ba94-e647a2b6fcc8 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:01.605310 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b820f2-1c9b-4979-8fe2-28bc578eab02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:01.606087 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b820f2-1c9b-4979-8fe2-28bc578eab02 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:01.606307 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b820f2-1c9b-4979-8fe2-28bc578eab02 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:01.606568 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b820f2-1c9b-4979-8fe2-28bc578eab02 None admin] PATH_INFO: `/v3/services` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:01.607316 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b820f2-1c9b-4979-8fe2-28bc578eab02 None admin] RBAC: Adding query filter params (name=network) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:01.607630 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b820f2-1c9b-4979-8fe2-28bc578eab02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:01.609834 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b820f2-1c9b-4979-8fe2-28bc578eab02 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:01.620824 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 354/711] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Jul 17 16:41:01 2022] GET /identity/v3/services?name=network => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:01.625620 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b820f2-1c9b-4979-8fe2-28bc578eab02 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:01.768073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51757852-fd1e-4706-9533-583dc9fd2a10 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:01.803157 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b820f2-1c9b-4979-8fe2-28bc578eab02 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:01.839643 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3bc37-d3bf-4ee7-94e0-08a856529b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:01.840336 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3bc37-d3bf-4ee7-94e0-08a856529b96 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:01.840695 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3bc37-d3bf-4ee7-94e0-08a856529b96 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:01.840988 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3bc37-d3bf-4ee7-94e0-08a856529b96 None admin] PATH_INFO: `/v3/projects` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:01.841756 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b3bc37-d3bf-4ee7-94e0-08a856529b96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:01.842071 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b3bc37-d3bf-4ee7-94e0-08a856529b96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:01.843696 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b3bc37-d3bf-4ee7-94e0-08a856529b96 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:01.844730 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667dfd8f-c0f8-45e8-b761-995af88d676e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:01.845500 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667dfd8f-c0f8-45e8-b761-995af88d676e None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:01.845873 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667dfd8f-c0f8-45e8-b761-995af88d676e None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:01.846153 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667dfd8f-c0f8-45e8-b761-995af88d676e None admin] PATH_INFO: `/v3/services` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:01.847012 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667dfd8f-c0f8-45e8-b761-995af88d676e None admin] RBAC: Adding query filter params (type=network) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:01.847335 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667dfd8f-c0f8-45e8-b761-995af88d676e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:01.848932 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667dfd8f-c0f8-45e8-b761-995af88d676e None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:01.856425 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:01.856775 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 355/712] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Jul 17 16:41:01 2022] GET /identity/v3/services?type=network => generated 124 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:41:01.857752 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 358/713] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:41:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:41:01.859571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4cdcfa-367e-4f0e-b9f1-09e6767c9e99 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:01.860025 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4cdcfa-367e-4f0e-b9f1-09e6767c9e99 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:01.860331 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4cdcfa-367e-4f0e-b9f1-09e6767c9e99 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:01.861934 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 356/714] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:01.868063 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ece581-9b33-4595-9215-07933dedfd83 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:01.868370 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ece581-9b33-4595-9215-07933dedfd83 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:01.868690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ece581-9b33-4595-9215-07933dedfd83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:01.883902 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4cdcfa-367e-4f0e-b9f1-09e6767c9e99 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:01.897250 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-f6ece581-9b33-4595-9215-07933dedfd83 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:01.900238 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-f6ece581-9b33-4595-9215-07933dedfd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:02.020911 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4cdcfa-367e-4f0e-b9f1-09e6767c9e99 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:02.081186 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:02.098008 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 359/715] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 17 16:41:02.121790 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46896eab-5cf0-4daf-82fb-df1c1933021f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:02.122401 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46896eab-5cf0-4daf-82fb-df1c1933021f None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:02.122530 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ece581-9b33-4595-9215-07933dedfd83 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:02.122980 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46896eab-5cf0-4daf-82fb-df1c1933021f None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:02.123266 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46896eab-5cf0-4daf-82fb-df1c1933021f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:02.124136 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46896eab-5cf0-4daf-82fb-df1c1933021f None admin] RBAC: Adding query filter params (role.id=1025051975f64f689effa58dc4259e36, scope.project.id=927057f9a5af4833aeac9992203a223c, user.id=03244087f1a34bddb11ea46aba11a148) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:02.124658 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46896eab-5cf0-4daf-82fb-df1c1933021f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:02.126600 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46896eab-5cf0-4daf-82fb-df1c1933021f None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:02.144972 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 357/716] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:41:01 2022] GET /identity/v3/role_assignments?user.id=03244087f1a34bddb11ea46aba11a148&scope.project.id=927057f9a5af4833aeac9992203a223c&role.id=1025051975f64f689effa58dc4259e36 => generated 259 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:02.323829 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ece581-9b33-4595-9215-07933dedfd83 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:02.382022 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0b5cc-2ac2-4343-919b-5dd021eef026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:02.383406 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0b5cc-2ac2-4343-919b-5dd021eef026 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:02.383928 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0b5cc-2ac2-4343-919b-5dd021eef026 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:02.384513 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa0b5cc-2ac2-4343-919b-5dd021eef026 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:02.387932 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa0b5cc-2ac2-4343-919b-5dd021eef026 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:02.390442 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa0b5cc-2ac2-4343-919b-5dd021eef026 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:02.421935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-efa0b5cc-2ac2-4343-919b-5dd021eef026 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 17 16:41:02.421935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:02.421935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 17 16:41:02.421935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 17 16:41:02.421935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 17 16:41:02.421935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 17 16:41:02.421935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 17 16:41:02.421935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:02.421935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 17 16:41:02.421935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:02.421935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 17 16:41:02.421935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:02.421935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:02.421935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:02.421935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:02.421935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:02.422935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:02.422935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:02.422935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:02.422935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:02.422935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:02.422935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:02.422935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:02.422935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 17 16:41:02.422935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 17 16:41:02.422935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 17 16:41:02.422935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 17 16:41:02.422935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:02.422935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:02.422935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:02.422935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:02.422935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:02.423791 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:02.423791 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:02.423791 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:02.423791 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:02.423791 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:02.423791 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:02.423791 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:02.423791 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:02.423791 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:02.423791 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:02.423791 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:02.423791 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 17 16:41:02.423791 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:02.423791 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 17 16:41:02.423791 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:02.425145 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 360/717] 10.222.0.40 () {56 vars in 1129 bytes} [Sun Jul 17 16:41:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:02.431187 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46896eab-5cf0-4daf-82fb-df1c1933021f None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:02.568009 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:02.656245 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46896eab-5cf0-4daf-82fb-df1c1933021f None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:02.746653 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8577b8e6-8c77-4184-b9a9-3ded074545ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:02.747825 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577b8e6-8c77-4184-b9a9-3ded074545ba None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:02.748316 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577b8e6-8c77-4184-b9a9-3ded074545ba None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:02.748846 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8577b8e6-8c77-4184-b9a9-3ded074545ba None admin] PATH_INFO: `/v3/services` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:02.750954 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8577b8e6-8c77-4184-b9a9-3ded074545ba None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:02.751532 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8577b8e6-8c77-4184-b9a9-3ded074545ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:02.754151 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8577b8e6-8c77-4184-b9a9-3ded074545ba None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:02.763821 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 358/718] 10.222.0.40 () {56 vars in 1124 bytes} [Sun Jul 17 16:41:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:02.770204 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0b5cc-2ac2-4343-919b-5dd021eef026 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:02.885157 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:03.036150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa0b5cc-2ac2-4343-919b-5dd021eef026 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:03.118244 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5b4d4-5d89-4c09-aad1-bfdc71a18dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:03.119373 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5b4d4-5d89-4c09-aad1-bfdc71a18dc5 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:03.119887 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5b4d4-5d89-4c09-aad1-bfdc71a18dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:03.120404 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5b4d4-5d89-4c09-aad1-bfdc71a18dc5 None admin] PATH_INFO: `/v3/services` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:03.121770 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a5b4d4-5d89-4c09-aad1-bfdc71a18dc5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:03.122326 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a5b4d4-5d89-4c09-aad1-bfdc71a18dc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:03.125491 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a5b4d4-5d89-4c09-aad1-bfdc71a18dc5 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:03.134042 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 361/719] 10.222.0.40 () {56 vars in 1124 bytes} [Sun Jul 17 16:41:02 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:03.599248 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1be192-c645-4326-9e1a-d9fa6ca24d45 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:03.599771 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1be192-c645-4326-9e1a-d9fa6ca24d45 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:03.600196 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1be192-c645-4326-9e1a-d9fa6ca24d45 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:03.605205 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 359/720] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:03.611763 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b858fa6-f80c-41b5-a4b3-c7c28f5d55f2 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:03.612117 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b858fa6-f80c-41b5-a4b3-c7c28f5d55f2 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:03.612465 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b858fa6-f80c-41b5-a4b3-c7c28f5d55f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:03.646582 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-5b858fa6-f80c-41b5-a4b3-c7c28f5d55f2 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:03.651323 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-5b858fa6-f80c-41b5-a4b3-c7c28f5d55f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:03.794778 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 362/721] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:41:03.848983 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1be192-c645-4326-9e1a-d9fa6ca24d45 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:03.975264 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b71a77-18ba-4c66-9452-52340327586a None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:03.975514 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b71a77-18ba-4c66-9452-52340327586a None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:03.975754 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b71a77-18ba-4c66-9452-52340327586a None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:03.981700 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 363/722] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:03.992942 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887d204-035b-45f5-8325-f5fefe71f4c7 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:03.993251 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887d204-035b-45f5-8325-f5fefe71f4c7 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:03.993603 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887d204-035b-45f5-8325-f5fefe71f4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:04.031337 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1be192-c645-4326-9e1a-d9fa6ca24d45 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:04.045131 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-2887d204-035b-45f5-8325-f5fefe71f4c7 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:04.046507 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-2887d204-035b-45f5-8325-f5fefe71f4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:04.100654 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:04.111536 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:04.118079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1761038f-ec11-487d-9934-7860292b3265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:04.119851 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761038f-ec11-487d-9934-7860292b3265 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:04.119851 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761038f-ec11-487d-9934-7860292b3265 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:04.119851 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1761038f-ec11-487d-9934-7860292b3265 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:04.145810 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1761038f-ec11-487d-9934-7860292b3265 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:41:04.156729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-1761038f-ec11-487d-9934-7860292b3265 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:41:04.156729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:04.156729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:04.156729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:04.156729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:04.156729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:04.156729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:04.156729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:04.156729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:04.156729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:04.156729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:04.156729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:04.156729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:41:04.156729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:41:04.156729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:41:04.156729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:41:04.157763 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:41:04.157763 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:41:04.157763 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:41:04.157763 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:41:04.157763 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:04.157763 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:04.157763 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:04.157763 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:04.157763 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:04.157763 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:04.157763 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:04.157763 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:04.157763 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:04.157763 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:04.157763 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:04.157763 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:04.158582 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:04.158582 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:04.158582 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:04.158582 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:04.158582 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:41:04.158582 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:41:04.158582 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:41:04.158582 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:41:04.158582 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:41:04.158582 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:41:04.158582 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:41:04.158582 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:04.161309 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 360/723] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 17 16:41:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:04.180007 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1761038f-ec11-487d-9934-7860292b3265 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:04.198578 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 364/724] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 17 16:41:04.225986 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2887d204-035b-45f5-8325-f5fefe71f4c7 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:04.381931 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2887d204-035b-45f5-8325-f5fefe71f4c7 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:04.458377 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1761038f-ec11-487d-9934-7860292b3265 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:04.480708 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3cb32a-f8a9-4d8c-a5db-a39d8ecd7c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:04.481780 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3cb32a-f8a9-4d8c-a5db-a39d8ecd7c7a None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:04.482258 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3cb32a-f8a9-4d8c-a5db-a39d8ecd7c7a None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:04.482727 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3cb32a-f8a9-4d8c-a5db-a39d8ecd7c7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:04.509975 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3cb32a-f8a9-4d8c-a5db-a39d8ecd7c7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:41:04.518824 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-aa3cb32a-f8a9-4d8c-a5db-a39d8ecd7c7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:41:04.518824 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:04.518824 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:04.518824 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:04.518824 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:04.518824 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:04.518824 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:04.518824 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:04.518824 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:04.518824 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:04.518824 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:04.518824 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:04.518824 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:41:04.518824 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:41:04.518824 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:41:04.518824 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:41:04.519894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:41:04.519894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:41:04.519894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:41:04.519894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:41:04.519894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:04.519894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:04.519894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:04.519894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:04.519894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:04.519894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:04.519894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:04.519894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:04.519894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:04.519894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:04.519894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:04.519894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:04.520716 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:04.520716 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:04.520716 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:04.520716 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:04.520716 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:41:04.520716 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:41:04.520716 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:41:04.520716 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:41:04.520716 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:41:04.520716 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:41:04.520716 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:41:04.520716 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:04.520716 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 365/725] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:41:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:04.527453 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdec5b-a1a7-492f-ae8c-2e789a77cf74 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:04.527821 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdec5b-a1a7-492f-ae8c-2e789a77cf74 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:04.528113 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdec5b-a1a7-492f-ae8c-2e789a77cf74 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:04.529835 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 366/726] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:04.531140 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fdec5b-a1a7-492f-ae8c-2e789a77cf74 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:04.542622 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7f2b78-16e5-4498-9ee8-e2314e9ba129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:04.543310 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f2b78-16e5-4498-9ee8-e2314e9ba129 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:04.543595 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f2b78-16e5-4498-9ee8-e2314e9ba129 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:04.543874 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f2b78-16e5-4498-9ee8-e2314e9ba129 None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:04.544645 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7f2b78-16e5-4498-9ee8-e2314e9ba129 None admin] RBAC: Adding query filter params (name=service) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:04.544997 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7f2b78-16e5-4498-9ee8-e2314e9ba129 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:04.547084 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7f2b78-16e5-4498-9ee8-e2314e9ba129 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:04.556478 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 361/727] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 17 16:41:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:41:04.558003 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c67922e-7941-4b0f-9a9c-7841ee65a00c None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:04.558300 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c67922e-7941-4b0f-9a9c-7841ee65a00c None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:04.558584 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c67922e-7941-4b0f-9a9c-7841ee65a00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:04.579499 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-5c67922e-7941-4b0f-9a9c-7841ee65a00c None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:04.580743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-5c67922e-7941-4b0f-9a9c-7841ee65a00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:04.598137 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:04.708966 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:04.754258 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fdec5b-a1a7-492f-ae8c-2e789a77cf74 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:04.760913 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 362/728] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:41:04.762935 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c67922e-7941-4b0f-9a9c-7841ee65a00c None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:04.817198 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39cd9e0-8488-4f24-82e5-a648053374e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:04.817944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cd9e0-8488-4f24-82e5-a648053374e6 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:04.818327 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cd9e0-8488-4f24-82e5-a648053374e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:04.818862 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cd9e0-8488-4f24-82e5-a648053374e6 None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:04.819658 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39cd9e0-8488-4f24-82e5-a648053374e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:04.819997 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39cd9e0-8488-4f24-82e5-a648053374e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:04.821452 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39cd9e0-8488-4f24-82e5-a648053374e6 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:04.838230 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 367/729] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:41:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:04.838230 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b67f0-c0ee-4810-a5c4-bbd3943547dc None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:04.838230 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b67f0-c0ee-4810-a5c4-bbd3943547dc None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:04.838230 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b67f0-c0ee-4810-a5c4-bbd3943547dc None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:04.838230 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 368/730] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:04.838959 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b67f0-c0ee-4810-a5c4-bbd3943547dc None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:04.932026 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c67922e-7941-4b0f-9a9c-7841ee65a00c None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:04.973130 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9aa07d-ef03-40c5-8ff3-e1c1e238bb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:04.973945 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9aa07d-ef03-40c5-8ff3-e1c1e238bb87 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:04.974228 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9aa07d-ef03-40c5-8ff3-e1c1e238bb87 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:04.974506 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9aa07d-ef03-40c5-8ff3-e1c1e238bb87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:04.975948 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:04.987062 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9aa07d-ef03-40c5-8ff3-e1c1e238bb87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:04.988571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9aa07d-ef03-40c5-8ff3-e1c1e238bb87 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:04.990268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b67f0-c0ee-4810-a5c4-bbd3943547dc None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:05.009463 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-cf9aa07d-ef03-40c5-8ff3-e1c1e238bb87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:05.009463 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:05.009463 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:05.009463 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:05.009463 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:05.009463 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:05.009463 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:05.009463 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:05.009463 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:05.009463 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:05.009463 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:05.009463 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:05.009463 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:41:05.009463 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:41:05.009463 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:41:05.011076 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:41:05.011076 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:05.011076 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:05.011076 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:05.011076 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:05.011076 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:05.011076 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:05.011076 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:05.011076 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:05.011076 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:05.011076 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:05.011076 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:05.011076 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:05.011076 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:05.011076 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:05.011076 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:05.011076 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:05.012721 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:41:05.012721 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:41:05.012721 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:05.012721 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:05.015658 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 363/731] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:05.017343 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa248f94-7c34-4d3e-95fb-91b01e98929b None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:05.017616 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa248f94-7c34-4d3e-95fb-91b01e98929b None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:05.017885 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa248f94-7c34-4d3e-95fb-91b01e98929b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:05.036908 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-aa248f94-7c34-4d3e-95fb-91b01e98929b None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:05.038134 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-aa248f94-7c34-4d3e-95fb-91b01e98929b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:05.039547 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2de90-2255-45a2-948c-9b030957393b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:05.040666 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2de90-2255-45a2-948c-9b030957393b None admin] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:05.042071 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2de90-2255-45a2-948c-9b030957393b None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:05.042495 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a2de90-2255-45a2-948c-9b030957393b None admin] PATH_INFO: `/v3/services` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:05.043396 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a2de90-2255-45a2-948c-9b030957393b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:05.044999 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a2de90-2255-45a2-948c-9b030957393b None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:05.068996 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:05.110297 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 369/732] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Jul 17 16:41:04 2022] POST /identity/v3/services => generated 240 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:41:05.112375 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2de90-2255-45a2-948c-9b030957393b None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:05.256207 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 364/733] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:05 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:41:05.257142 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa248f94-7c34-4d3e-95fb-91b01e98929b None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:05.354107 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a2de90-2255-45a2-948c-9b030957393b None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:05.404120 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d3153-7e1c-4de3-97c5-79ff3e7bc920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:05.409946 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d3153-7e1c-4de3-97c5-79ff3e7bc920 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:05.409946 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d3153-7e1c-4de3-97c5-79ff3e7bc920 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:05.409946 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d3153-7e1c-4de3-97c5-79ff3e7bc920 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:05.427130 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d3153-7e1c-4de3-97c5-79ff3e7bc920 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:05.429745 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d3153-7e1c-4de3-97c5-79ff3e7bc920 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:05.448986 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa248f94-7c34-4d3e-95fb-91b01e98929b None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:05.460881 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-e22d3153-7e1c-4de3-97c5-79ff3e7bc920 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:05.460881 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:05.460881 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:05.460881 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:05.460881 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:05.460881 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:05.460881 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:05.460881 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:05.460881 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:05.460881 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:05.460881 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:05.460881 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:05.460881 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:41:05.460881 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:41:05.460881 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:41:05.461871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:41:05.461871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:05.461871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:05.461871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:05.461871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:05.461871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:05.461871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:05.461871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:05.461871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:05.461871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:05.461871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:05.461871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:05.461871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:05.461871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:05.461871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:05.461871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:05.461871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:05.462792 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:41:05.462792 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:41:05.462792 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:05.462792 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:05.462792 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 370/734] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:05.462792 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d3153-7e1c-4de3-97c5-79ff3e7bc920 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:05.462792 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:05.525808 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa89d1-630b-40c6-bc79-9277b9cec1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:05.527085 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa89d1-630b-40c6-bc79-9277b9cec1fb None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:05.527760 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa89d1-630b-40c6-bc79-9277b9cec1fb None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:05.528201 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa89d1-630b-40c6-bc79-9277b9cec1fb None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:05.529677 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefa89d1-630b-40c6-bc79-9277b9cec1fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:05.530339 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefa89d1-630b-40c6-bc79-9277b9cec1fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:05.533931 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefa89d1-630b-40c6-bc79-9277b9cec1fb None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:05.558939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 365/735] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:41:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:05.560927 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa89d1-630b-40c6-bc79-9277b9cec1fb None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:05.609291 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:05.625588 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d3153-7e1c-4de3-97c5-79ff3e7bc920 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:05.715906 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca5eca-443f-4abc-8ee6-6fc69aeae8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:05.717363 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca5eca-443f-4abc-8ee6-6fc69aeae8d2 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:05.718076 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca5eca-443f-4abc-8ee6-6fc69aeae8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:05.718371 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca5eca-443f-4abc-8ee6-6fc69aeae8d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:05.740187 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cca5eca-443f-4abc-8ee6-6fc69aeae8d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:41:05.756196 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-0cca5eca-443f-4abc-8ee6-6fc69aeae8d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:41:05.756196 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:05.756196 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:05.756196 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:05.756196 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:05.756196 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:05.756196 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:05.756196 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:05.756196 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:05.756196 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:05.756196 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:05.756196 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:05.756196 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:41:05.756196 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:41:05.756196 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:41:05.756196 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:41:05.758143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:41:05.758143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:41:05.758143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:41:05.758143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:41:05.758143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:05.758143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:05.758143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:05.758143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:05.758143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:05.758143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:05.758143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:05.758143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:05.758143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:05.758143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:05.758143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:05.758143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:05.760447 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:05.760447 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:05.760447 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:05.760447 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:05.760447 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:41:05.760447 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:41:05.760447 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:41:05.760447 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:41:05.760447 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:41:05.760447 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:41:05.760447 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:41:05.760447 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:05.760447 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 371/736] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:41:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:05.760447 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca5eca-443f-4abc-8ee6-6fc69aeae8d2 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:05.797708 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa89d1-630b-40c6-bc79-9277b9cec1fb None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:05.869851 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b793c86-c45d-43c0-97a2-591d226d1f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:05.870877 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b793c86-c45d-43c0-97a2-591d226d1f1c None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:05.871441 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b793c86-c45d-43c0-97a2-591d226d1f1c None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:05.871947 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b793c86-c45d-43c0-97a2-591d226d1f1c None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:05.873266 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b793c86-c45d-43c0-97a2-591d226d1f1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:05.873816 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b793c86-c45d-43c0-97a2-591d226d1f1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:05.877996 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b793c86-c45d-43c0-97a2-591d226d1f1c None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:05.894715 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:05.912682 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 366/737] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:41:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:05.915290 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b793c86-c45d-43c0-97a2-591d226d1f1c None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:05.981095 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca5eca-443f-4abc-8ee6-6fc69aeae8d2 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:06.048891 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8adc9-a8fc-40d8-82ad-b205f29c579e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:06.050726 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8adc9-a8fc-40d8-82ad-b205f29c579e None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:06.051502 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8adc9-a8fc-40d8-82ad-b205f29c579e None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:06.052319 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8adc9-a8fc-40d8-82ad-b205f29c579e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:06.069357 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b8adc9-a8fc-40d8-82ad-b205f29c579e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:06.072591 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b8adc9-a8fc-40d8-82ad-b205f29c579e None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:06.081222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:06.104610 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-a9b8adc9-a8fc-40d8-82ad-b205f29c579e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 17 16:41:06.104610 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:06.104610 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:06.104610 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:06.104610 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:06.104610 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:06.104610 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:06.104610 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:06.104610 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:06.104610 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:06.104610 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:06.104610 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:06.104610 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:41:06.104610 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:41:06.104610 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:41:06.104610 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:41:06.106536 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:06.106536 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:06.106536 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:41:06.106536 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:41:06.106536 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:41:06.106536 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:41:06.106536 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:06.106536 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:06.106536 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:06.106536 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:06.106536 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:06.106536 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:06.106536 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:06.106536 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:06.106536 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:06.106536 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:06.106536 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:06.106536 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:06.108247 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:06.108247 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:06.108247 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:41:06.108247 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:41:06.108247 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:41:06.108247 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:41:06.108247 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:41:06.108247 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:41:06.108247 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 17 16:41:06.108247 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:06.114596 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 372/738] 10.222.0.40 () {56 vars in 1117 bytes} [Sun Jul 17 16:41:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:06.116615 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8adc9-a8fc-40d8-82ad-b205f29c579e None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:06.117741 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b793c86-c45d-43c0-97a2-591d226d1f1c None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:06.190320 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8f425-2226-4520-a855-b19f4a57f50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:06.191343 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8f425-2226-4520-a855-b19f4a57f50a None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:06.191845 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8f425-2226-4520-a855-b19f4a57f50a None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:06.192335 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8f425-2226-4520-a855-b19f4a57f50a None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:06.194140 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d8f425-2226-4520-a855-b19f4a57f50a None admin] RBAC: Adding query filter params (name=admin) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:06.194692 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d8f425-2226-4520-a855-b19f4a57f50a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:06.197197 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d8f425-2226-4520-a855-b19f4a57f50a None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:06.222338 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 367/739] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:41:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:06.223909 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8f425-2226-4520-a855-b19f4a57f50a None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:06.321336 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8adc9-a8fc-40d8-82ad-b205f29c579e None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:06.366252 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:06.386893 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c7771-f99b-4765-82ac-e14580ba4244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:06.388274 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c7771-f99b-4765-82ac-e14580ba4244 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:06.388798 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c7771-f99b-4765-82ac-e14580ba4244 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:06.389308 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c7771-f99b-4765-82ac-e14580ba4244 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:06.419770 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8f425-2226-4520-a855-b19f4a57f50a None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:06.488072 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6f1e7-58e5-4d69-93fe-6861dd2c89a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:06.489148 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6f1e7-58e5-4d69-93fe-6861dd2c89a2 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:06.489643 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6f1e7-58e5-4d69-93fe-6861dd2c89a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:06.490133 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6f1e7-58e5-4d69-93fe-6861dd2c89a2 None admin] PATH_INFO: `/v3/users` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:06.491810 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b6f1e7-58e5-4d69-93fe-6861dd2c89a2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:06.492368 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b6f1e7-58e5-4d69-93fe-6861dd2c89a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:06.495203 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b6f1e7-58e5-4d69-93fe-6861dd2c89a2 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:06.516732 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:06.538637 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 368/740] 10.222.0.40 () {56 vars in 1149 bytes} [Sun Jul 17 16:41:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:06.538637 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6f1e7-58e5-4d69-93fe-6861dd2c89a2 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:06.539034 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c7771-f99b-4765-82ac-e14580ba4244 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:06.539191 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c7771-f99b-4765-82ac-e14580ba4244 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:06.557795 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-e41c7771-f99b-4765-82ac-e14580ba4244 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 17 16:41:06.557795 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:06.557795 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:06.557795 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:06.557795 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:06.557795 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:06.557795 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:06.557795 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:06.557795 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:06.557795 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:06.557795 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:06.557795 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:06.557795 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:41:06.557795 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:41:06.557795 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:41:06.557795 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:41:06.567161 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:06.567161 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:06.567161 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:41:06.567161 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:41:06.567161 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:41:06.567161 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:41:06.567161 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:06.567161 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:06.567161 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:06.567161 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:06.567161 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:06.567161 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:06.567161 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:06.567161 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:06.567161 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:06.567161 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:06.567161 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:06.567161 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:06.570054 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:06.570054 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:06.570054 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:41:06.570054 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:41:06.570054 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:41:06.570054 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:41:06.570054 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:41:06.570054 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:41:06.570054 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 17 16:41:06.570054 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:06.570054 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 373/741] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:41:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 444 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:06.570054 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29c4b6-9f5b-4715-b668-0ca38b7bf83b None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:06.570054 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29c4b6-9f5b-4715-b668-0ca38b7bf83b None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:06.574173 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29c4b6-9f5b-4715-b668-0ca38b7bf83b None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:06.574173 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 374/742] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:06.574173 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e29c4b6-9f5b-4715-b668-0ca38b7bf83b None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:06.711700 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e29c4b6-9f5b-4715-b668-0ca38b7bf83b None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:06.725832 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6f1e7-58e5-4d69-93fe-6861dd2c89a2 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:06.764228 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47b784-3871-4bd4-921e-ac4809056683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:06.765566 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47b784-3871-4bd4-921e-ac4809056683 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:06.765872 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47b784-3871-4bd4-921e-ac4809056683 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:06.766117 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47b784-3871-4bd4-921e-ac4809056683 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:06.777007 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db4ef7-93ee-4f1e-a4a5-d7568910db90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:06.777553 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47b784-3871-4bd4-921e-ac4809056683 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:06.778880 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db4ef7-93ee-4f1e-a4a5-d7568910db90 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:06.779042 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47b784-3871-4bd4-921e-ac4809056683 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:06.779806 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db4ef7-93ee-4f1e-a4a5-d7568910db90 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:06.781113 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db4ef7-93ee-4f1e-a4a5-d7568910db90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:06.800485 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-be47b784-3871-4bd4-921e-ac4809056683 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:06.800485 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:06.800485 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:06.800485 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:06.800485 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:06.800485 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:06.800485 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:06.800485 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:06.800485 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:06.800485 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:06.800485 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:06.800485 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:06.800485 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:41:06.800485 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:41:06.800485 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:41:06.802758 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:41:06.802758 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:06.802758 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:06.802758 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:06.802758 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:06.802758 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:06.802758 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:06.802758 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:06.802758 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:06.802758 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:06.802758 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:06.802758 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:06.802758 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:06.802758 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:06.802758 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:06.802758 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:06.802758 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:06.804462 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:41:06.804462 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:41:06.804462 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:06.804462 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:06.804462 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 369/743] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:06.804462 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47b784-3871-4bd4-921e-ac4809056683 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:06.869413 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:06.874815 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:06.906586 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11db4ef7-93ee-4f1e-a4a5-d7568910db90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:06.908954 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11db4ef7-93ee-4f1e-a4a5-d7568910db90 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:06.940419 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47b784-3871-4bd4-921e-ac4809056683 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:06.999647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75577b-ec11-423d-917c-b99bd5bc3545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:07.000775 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75577b-ec11-423d-917c-b99bd5bc3545 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:07.001272 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75577b-ec11-423d-917c-b99bd5bc3545 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:07.001675 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75577b-ec11-423d-917c-b99bd5bc3545 None admin] PATH_INFO: `/v3/users` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:07.003164 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d75577b-ec11-423d-917c-b99bd5bc3545 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:07.003601 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d75577b-ec11-423d-917c-b99bd5bc3545 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:07.005959 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d75577b-ec11-423d-917c-b99bd5bc3545 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:07.037060 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 370/744] 10.222.0.40 () {56 vars in 1135 bytes} [Sun Jul 17 16:41:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:07.038710 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863d8bd-15f4-4d35-8a58-c7e0500fe8a6 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:07.039098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863d8bd-15f4-4d35-8a58-c7e0500fe8a6 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:07.039492 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3863d8bd-15f4-4d35-8a58-c7e0500fe8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:07.060077 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 375/745] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:41:06 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 17 16:41:07.062278 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db4ef7-93ee-4f1e-a4a5-d7568910db90 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:07.066424 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-3863d8bd-15f4-4d35-8a58-c7e0500fe8a6 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:07.068517 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-3863d8bd-15f4-4d35-8a58-c7e0500fe8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:07.168816 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 371/746] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:07 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 17 16:41:07.168816 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863d8bd-15f4-4d35-8a58-c7e0500fe8a6 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:07.214056 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:07.220180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:07.243737 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db4ef7-93ee-4f1e-a4a5-d7568910db90 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:07.299337 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a4298e-a2b4-46f0-99c7-7e95c4a31ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:07.300290 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4298e-a2b4-46f0-99c7-7e95c4a31ec8 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:07.300753 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4298e-a2b4-46f0-99c7-7e95c4a31ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:07.301245 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a4298e-a2b4-46f0-99c7-7e95c4a31ec8 None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:07.302487 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a4298e-a2b4-46f0-99c7-7e95c4a31ec8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:07.302656 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3863d8bd-15f4-4d35-8a58-c7e0500fe8a6 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:07.302976 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a4298e-a2b4-46f0-99c7-7e95c4a31ec8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:07.305253 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a4298e-a2b4-46f0-99c7-7e95c4a31ec8 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:07.334169 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 376/747] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:41:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:07.334169 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a4298e-a2b4-46f0-99c7-7e95c4a31ec8 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:07.373607 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc0eee9-93ca-4700-99a6-6f633df075f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:07.375708 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0eee9-93ca-4700-99a6-6f633df075f1 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:07.376430 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0eee9-93ca-4700-99a6-6f633df075f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:07.376958 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0eee9-93ca-4700-99a6-6f633df075f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:07.498751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a4298e-a2b4-46f0-99c7-7e95c4a31ec8 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:07.502279 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc0eee9-93ca-4700-99a6-6f633df075f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:07.505073 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc0eee9-93ca-4700-99a6-6f633df075f1 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:07.565558 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b49ea-2a12-4eaf-aafc-05b673a59793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:07.566812 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b49ea-2a12-4eaf-aafc-05b673a59793 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:07.567302 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b49ea-2a12-4eaf-aafc-05b673a59793 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:07.567776 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b49ea-2a12-4eaf-aafc-05b673a59793 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:07.581357 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:07.588299 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094b49ea-2a12-4eaf-aafc-05b673a59793 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:07.591145 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094b49ea-2a12-4eaf-aafc-05b673a59793 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:07.595558 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:07.628050 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-094b49ea-2a12-4eaf-aafc-05b673a59793 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:07.628050 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:07.628050 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:07.628050 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:07.628050 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:07.628050 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:07.628050 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:07.628050 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:07.628050 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:07.628050 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:07.628050 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:07.628050 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:07.628050 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:41:07.628050 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:41:07.628050 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:41:07.629432 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:41:07.629432 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:07.629432 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:07.629432 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:07.629432 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:07.629432 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:07.629432 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:07.629432 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:07.629432 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:07.629432 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:07.629432 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:07.629432 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:07.629432 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:07.629432 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:07.629432 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:07.629432 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:07.629432 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:07.630326 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:41:07.630326 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:41:07.630326 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:07.630326 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:07.637181 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 377/748] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:07.645106 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b49ea-2a12-4eaf-aafc-05b673a59793 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:07.703128 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 372/749] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:41:07 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 17 16:41:07.704687 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc0eee9-93ca-4700-99a6-6f633df075f1 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:07.796593 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b49ea-2a12-4eaf-aafc-05b673a59793 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:07.853894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c01c43-6d38-4af1-adc5-3097e0ec4200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:07.854801 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c01c43-6d38-4af1-adc5-3097e0ec4200 None admin] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:07.855262 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c01c43-6d38-4af1-adc5-3097e0ec4200 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:07.855714 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c01c43-6d38-4af1-adc5-3097e0ec4200 None admin] PATH_INFO: `/v3/services` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:07.856813 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c01c43-6d38-4af1-adc5-3097e0ec4200 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:07.859660 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c01c43-6d38-4af1-adc5-3097e0ec4200 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:07.869235 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc0eee9-93ca-4700-99a6-6f633df075f1 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:07.906112 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc05290-5503-439a-829c-6737bc005e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:07.906959 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc05290-5503-439a-829c-6737bc005e0f None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:07.907257 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc05290-5503-439a-829c-6737bc005e0f None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:07.907524 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc05290-5503-439a-829c-6737bc005e0f None admin] PATH_INFO: `/v3/users/nova` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:07.915913 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc05290-5503-439a-829c-6737bc005e0f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:07.917493 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc05290-5503-439a-829c-6737bc005e0f None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:07.919553 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:07.934260 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-9cc05290-5503-439a-829c-6737bc005e0f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 17 16:41:07.934260 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:07.934260 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:07.934260 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:07.934260 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:07.934260 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:07.934260 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:07.934260 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:07.934260 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:07.934260 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:07.934260 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:07.934260 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:07.934260 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:41:07.934260 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:41:07.934260 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:41:07.934260 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:41:07.935450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:07.935450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:07.935450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:41:07.935450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:41:07.935450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:41:07.935450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:41:07.935450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:07.935450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:07.935450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:07.935450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:07.935450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:07.935450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:07.935450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:07.935450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:07.935450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:07.935450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:07.935450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:07.935450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:07.941550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:07.941550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:07.941550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:41:07.941550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:41:07.941550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:41:07.941550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:41:07.941550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:41:07.941550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:41:07.941550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 17 16:41:07.941550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:07.941550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 373/750] 10.222.0.40 () {56 vars in 1093 bytes} [Sun Jul 17 16:41:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:07.941550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc05290-5503-439a-829c-6737bc005e0f None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:07.948136 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 378/751] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Jul 17 16:41:07 2022] POST /identity/v3/services => generated 251 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:41:07.949328 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c01c43-6d38-4af1-adc5-3097e0ec4200 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:08.060314 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:08.098537 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc05290-5503-439a-829c-6737bc005e0f None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:08.149495 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c01c43-6d38-4af1-adc5-3097e0ec4200 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:08.157794 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f30a1df-b961-43bc-bd38-4ae339b1b589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:08.158728 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30a1df-b961-43bc-bd38-4ae339b1b589 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:08.159188 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30a1df-b961-43bc-bd38-4ae339b1b589 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:08.159600 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f30a1df-b961-43bc-bd38-4ae339b1b589 None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:08.161410 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f30a1df-b961-43bc-bd38-4ae339b1b589 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:08.161731 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f30a1df-b961-43bc-bd38-4ae339b1b589 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:08.163157 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f30a1df-b961-43bc-bd38-4ae339b1b589 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:08.182389 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 374/752] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:41:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:08.184220 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aabbc8-ac6a-46b5-85f3-82f153d770c6 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:08.184220 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aabbc8-ac6a-46b5-85f3-82f153d770c6 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:08.184434 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aabbc8-ac6a-46b5-85f3-82f153d770c6 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:08.186517 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 375/753] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:08.188138 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aabbc8-ac6a-46b5-85f3-82f153d770c6 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:08.204815 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ef2119-e90e-4cd8-96b7-0e793eb9723e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:08.205848 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef2119-e90e-4cd8-96b7-0e793eb9723e None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:08.206257 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef2119-e90e-4cd8-96b7-0e793eb9723e None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:08.206696 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef2119-e90e-4cd8-96b7-0e793eb9723e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:08.221174 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ef2119-e90e-4cd8-96b7-0e793eb9723e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:08.223042 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ef2119-e90e-4cd8-96b7-0e793eb9723e None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:08.245969 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-f3ef2119-e90e-4cd8-96b7-0e793eb9723e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:08.245969 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:08.245969 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:08.245969 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:08.245969 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:08.245969 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:08.245969 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:08.245969 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:08.245969 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:08.245969 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:08.245969 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:08.245969 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:08.245969 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:41:08.245969 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:41:08.245969 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:41:08.247668 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:41:08.247668 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:08.247668 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:08.247668 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:08.247668 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:08.247668 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:08.247668 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:08.247668 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:08.247668 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:08.247668 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:08.247668 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:08.247668 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:08.247668 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:08.247668 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:08.247668 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:08.247668 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:08.247668 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:08.258234 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:41:08.258234 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:41:08.258234 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:08.258234 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:08.258234 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 379/754] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:08.258234 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981f1b2-9e9a-4661-a626-474d7f9dd59d None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:08.258234 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981f1b2-9e9a-4661-a626-474d7f9dd59d None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:08.258234 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981f1b2-9e9a-4661-a626-474d7f9dd59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:08.260419 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:08.277078 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-a981f1b2-9e9a-4661-a626-474d7f9dd59d None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:08.279146 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-a981f1b2-9e9a-4661-a626-474d7f9dd59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:08.325186 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aabbc8-ac6a-46b5-85f3-82f153d770c6 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:08.376826 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 380/755] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:41:08.376826 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a981f1b2-9e9a-4661-a626-474d7f9dd59d None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:08.380729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d175af-d36d-4971-a8a4-ec2489b4d5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:08.381936 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d175af-d36d-4971-a8a4-ec2489b4d5f7 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:08.382480 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d175af-d36d-4971-a8a4-ec2489b4d5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:08.383068 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d175af-d36d-4971-a8a4-ec2489b4d5f7 None admin] PATH_INFO: `/v3/users` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:08.384212 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d175af-d36d-4971-a8a4-ec2489b4d5f7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:08.384854 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d175af-d36d-4971-a8a4-ec2489b4d5f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:08.387451 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d175af-d36d-4971-a8a4-ec2489b4d5f7 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:08.417128 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 376/756] 10.222.0.40 () {56 vars in 1133 bytes} [Sun Jul 17 16:41:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:08.419648 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d175af-d36d-4971-a8a4-ec2489b4d5f7 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:08.429867 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:08.543119 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a981f1b2-9e9a-4661-a626-474d7f9dd59d None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:08.571819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d175af-d36d-4971-a8a4-ec2489b4d5f7 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:08.588077 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:08.596087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd76a2f-53a0-4ad1-bef6-32dd552208b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:08.596959 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd76a2f-53a0-4ad1-bef6-32dd552208b5 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:08.597242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd76a2f-53a0-4ad1-bef6-32dd552208b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:08.597517 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd76a2f-53a0-4ad1-bef6-32dd552208b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:08.607221 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd76a2f-53a0-4ad1-bef6-32dd552208b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:08.609383 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd76a2f-53a0-4ad1-bef6-32dd552208b5 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:08.619107 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d6dcd-d844-4be1-bf93-8522e9919912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:08.619953 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d6dcd-d844-4be1-bf93-8522e9919912 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:08.620358 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d6dcd-d844-4be1-bf93-8522e9919912 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:08.620840 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d6dcd-d844-4be1-bf93-8522e9919912 None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:08.621855 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d6dcd-d844-4be1-bf93-8522e9919912 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:08.622310 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d6dcd-d844-4be1-bf93-8522e9919912 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:08.623933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d6dcd-d844-4be1-bf93-8522e9919912 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:08.626444 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-cbd76a2f-53a0-4ad1-bef6-32dd552208b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:41:08.626444 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:08.626444 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:08.626444 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:08.626444 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:08.626444 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:08.626444 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:08.626444 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:08.626444 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:08.626444 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:08.626444 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:08.626444 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:08.626444 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:41:08.626444 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:41:08.626444 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:41:08.628089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:41:08.628089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:08.628089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:08.628089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:08.628089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:08.628089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:08.628089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:08.628089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:08.628089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:08.628089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:08.628089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:08.628089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:08.628089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:08.628089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:08.628089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:08.628089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:08.628089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:08.631299 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:41:08.631299 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:41:08.631299 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:41:08.631299 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:41:08.631299 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:41:08.631299 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:41:08.631299 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:41:08.631299 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:08.631299 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 381/757] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:41:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:08.631299 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd76a2f-53a0-4ad1-bef6-32dd552208b5 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:08.648454 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 377/758] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:41:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:08.650481 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d6dcd-d844-4be1-bf93-8522e9919912 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:08.777745 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:08.795156 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd76a2f-53a0-4ad1-bef6-32dd552208b5 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:08.807599 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d6dcd-d844-4be1-bf93-8522e9919912 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:08.861611 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e4899-8358-4793-a071-efdd45f80c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:08.861889 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fbdbae-b864-4aec-bcd1-778c1149215f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:08.862425 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e4899-8358-4793-a071-efdd45f80c6b None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:08.862703 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e4899-8358-4793-a071-efdd45f80c6b None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:08.862989 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e4899-8358-4793-a071-efdd45f80c6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:08.863233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbdbae-b864-4aec-bcd1-778c1149215f None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:08.863779 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbdbae-b864-4aec-bcd1-778c1149215f None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:08.864207 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbdbae-b864-4aec-bcd1-778c1149215f None admin] PATH_INFO: `/v3/services/network` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:08.865879 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fbdbae-b864-4aec-bcd1-778c1149215f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:08.869099 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fbdbae-b864-4aec-bcd1-778c1149215f None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:08.878472 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e4899-8358-4793-a071-efdd45f80c6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:08.879946 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13e4899-8358-4793-a071-efdd45f80c6b None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:08.895865 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-f2fbdbae-b864-4aec-bcd1-778c1149215f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 17 16:41:08.895865 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:08.895865 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 17 16:41:08.895865 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 17 16:41:08.895865 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 17 16:41:08.895865 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 17 16:41:08.895865 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 17 16:41:08.895865 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:08.895865 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 17 16:41:08.895865 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:08.895865 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 17 16:41:08.895865 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:08.895865 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:08.895865 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:08.895865 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:08.895865 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:08.895865 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:08.897789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:08.897789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:08.897789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:08.897789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:08.897789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:08.897789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:08.897789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 17 16:41:08.897789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 17 16:41:08.897789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 17 16:41:08.897789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 17 16:41:08.897789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:08.897789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:08.897789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:08.897789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:08.897789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:08.897789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:08.908239 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:08.908239 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:08.908239 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:08.908239 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:08.908239 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:08.908239 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:08.908239 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:08.908239 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:08.908239 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:08.908239 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:08.908239 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 17 16:41:08.908239 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:08.908239 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 17 16:41:08.908239 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:08.908239 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 382/759] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:41:08 2022] GET /identity/v3/services/network => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:08.911387 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-c13e4899-8358-4793-a071-efdd45f80c6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:08.911387 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:08.911387 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:08.911387 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:08.911387 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:08.911387 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:08.911387 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:08.911387 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:08.911387 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:08.911387 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:08.911387 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:08.911387 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:08.911387 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:41:08.911387 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:41:08.911387 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:41:08.913200 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:41:08.913200 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:08.913200 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:08.913200 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:08.913200 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:08.913200 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:08.913200 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:08.913200 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:08.913200 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:08.913200 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:08.913200 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:08.913200 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:08.913200 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:08.913200 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:08.913200 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:08.913200 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:08.913200 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:08.915083 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:41:08.915083 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:41:08.915083 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:08.915083 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:08.915570 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fbdbae-b864-4aec-bcd1-778c1149215f None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:08.915709 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 378/760] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:08.915709 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e4899-8358-4793-a071-efdd45f80c6b None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:08.960270 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:09.066227 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fbdbae-b864-4aec-bcd1-778c1149215f None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:09.071333 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e4899-8358-4793-a071-efdd45f80c6b None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:09.126058 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d666d7c-fae1-4ca7-9749-0e0479515d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:09.127218 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d666d7c-fae1-4ca7-9749-0e0479515d22 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:09.128181 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d666d7c-fae1-4ca7-9749-0e0479515d22 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:09.128682 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d666d7c-fae1-4ca7-9749-0e0479515d22 None admin] PATH_INFO: `/v3/projects` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:09.130400 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d666d7c-fae1-4ca7-9749-0e0479515d22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:09.131114 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d666d7c-fae1-4ca7-9749-0e0479515d22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:09.133233 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a5fd87-abb8-47d7-93cd-31b0d421ac48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:09.134190 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d666d7c-fae1-4ca7-9749-0e0479515d22 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:09.134571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5fd87-abb8-47d7-93cd-31b0d421ac48 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:09.135076 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5fd87-abb8-47d7-93cd-31b0d421ac48 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:09.135586 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5fd87-abb8-47d7-93cd-31b0d421ac48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:09.148871 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a5fd87-abb8-47d7-93cd-31b0d421ac48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:09.151510 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a5fd87-abb8-47d7-93cd-31b0d421ac48 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:09.164874 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 383/761] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:41:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:09.164874 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d666d7c-fae1-4ca7-9749-0e0479515d22 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:09.176222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-09a5fd87-abb8-47d7-93cd-31b0d421ac48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:41:09.176222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:09.176222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:09.176222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:09.176222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:09.176222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:09.176222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:09.176222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:09.176222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:09.176222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:09.176222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:09.176222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:09.176222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:41:09.176222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:41:09.176222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:41:09.177182 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:41:09.177182 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:09.177182 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:09.177182 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:09.177182 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:09.177182 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:09.177182 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:09.177182 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:09.177182 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:09.177182 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:09.177182 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:09.177182 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:09.177182 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:09.177182 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:09.177182 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:09.177182 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:09.177182 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:09.178064 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:41:09.178064 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:41:09.178064 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:41:09.178064 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:41:09.178064 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:41:09.178064 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:41:09.178064 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:41:09.178064 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:09.181825 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 379/762] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:41:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:09.182918 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a5fd87-abb8-47d7-93cd-31b0d421ac48 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:09.201907 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:09.340536 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d666d7c-fae1-4ca7-9749-0e0479515d22 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:09.346512 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:09.365669 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a5fd87-abb8-47d7-93cd-31b0d421ac48 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:09.409938 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2595d2e-7e9f-4cbe-8986-33499d42ab03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:09.411121 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2595d2e-7e9f-4cbe-8986-33499d42ab03 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:09.411628 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2595d2e-7e9f-4cbe-8986-33499d42ab03 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:09.412136 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2595d2e-7e9f-4cbe-8986-33499d42ab03 None admin] PATH_INFO: `/v3/services` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:09.413530 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2595d2e-7e9f-4cbe-8986-33499d42ab03 None admin] RBAC: Adding query filter params (name=network) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:09.414099 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2595d2e-7e9f-4cbe-8986-33499d42ab03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:09.414256 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f438296-b3b2-4afb-8498-47264f576a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:09.414910 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f438296-b3b2-4afb-8498-47264f576a40 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:09.415217 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f438296-b3b2-4afb-8498-47264f576a40 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:09.415461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f438296-b3b2-4afb-8498-47264f576a40 None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:09.416196 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f438296-b3b2-4afb-8498-47264f576a40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:09.416500 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f438296-b3b2-4afb-8498-47264f576a40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:09.416741 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2595d2e-7e9f-4cbe-8986-33499d42ab03 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:09.417905 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f438296-b3b2-4afb-8498-47264f576a40 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:09.426003 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 384/763] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Jul 17 16:41:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:09.427575 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2595d2e-7e9f-4cbe-8986-33499d42ab03 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:09.446607 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 380/764] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:41:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:09.446607 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f438296-b3b2-4afb-8498-47264f576a40 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:09.584167 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:09.621634 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2595d2e-7e9f-4cbe-8986-33499d42ab03 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:09.627310 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f438296-b3b2-4afb-8498-47264f576a40 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:09.669857 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9568f47-2eeb-4db2-8995-7d64da9bb92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:09.670657 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9568f47-2eeb-4db2-8995-7d64da9bb92e None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:09.672046 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9568f47-2eeb-4db2-8995-7d64da9bb92e None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:09.672581 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9568f47-2eeb-4db2-8995-7d64da9bb92e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:09.673750 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9568f47-2eeb-4db2-8995-7d64da9bb92e None admin] RBAC: Adding query filter params (role.id=adb9c27c6e5246fdb0cf386358a822cd, scope.project.id=927057f9a5af4833aeac9992203a223c, user.id=37ed5f72dc8c4a5685bd1245cadcdf7e) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:09.674096 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9568f47-2eeb-4db2-8995-7d64da9bb92e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:09.675836 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9568f47-2eeb-4db2-8995-7d64da9bb92e None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:09.683388 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 385/765] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:41:09 2022] GET /identity/v3/role_assignments?user.id=37ed5f72dc8c4a5685bd1245cadcdf7e&scope.project.id=927057f9a5af4833aeac9992203a223c&role.id=adb9c27c6e5246fdb0cf386358a822cd => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:09.685050 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9568f47-2eeb-4db2-8995-7d64da9bb92e None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:09.709355 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8c9ce-3e06-4c72-9c6f-051324f9e791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:09.710545 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8c9ce-3e06-4c72-9c6f-051324f9e791 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:09.710987 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8c9ce-3e06-4c72-9c6f-051324f9e791 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:09.711456 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8c9ce-3e06-4c72-9c6f-051324f9e791 None admin] PATH_INFO: `/v3/projects` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:09.712951 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff8c9ce-3e06-4c72-9c6f-051324f9e791 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:09.713457 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff8c9ce-3e06-4c72-9c6f-051324f9e791 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:09.715960 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff8c9ce-3e06-4c72-9c6f-051324f9e791 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:09.740053 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 381/766] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:41:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:09.741630 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8c9ce-3e06-4c72-9c6f-051324f9e791 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:09.748197 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:09.949172 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9568f47-2eeb-4db2-8995-7d64da9bb92e None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:09.954095 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8c9ce-3e06-4c72-9c6f-051324f9e791 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:10.004899 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510aca96-312b-4366-b08f-2fbdce40c09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:10.006349 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510aca96-312b-4366-b08f-2fbdce40c09d None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:10.007192 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510aca96-312b-4366-b08f-2fbdce40c09d None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:10.007946 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510aca96-312b-4366-b08f-2fbdce40c09d None admin] PATH_INFO: `/v3/projects/service` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:10.025723 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510aca96-312b-4366-b08f-2fbdce40c09d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:10.029245 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510aca96-312b-4366-b08f-2fbdce40c09d None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:10.032139 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5440c4-025b-4796-89f9-c91ccc1565f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:10.032978 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5440c4-025b-4796-89f9-c91ccc1565f0 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:10.033340 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5440c4-025b-4796-89f9-c91ccc1565f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:10.033625 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5440c4-025b-4796-89f9-c91ccc1565f0 None admin] PATH_INFO: `/v3/services` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:10.034680 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5440c4-025b-4796-89f9-c91ccc1565f0 None admin] RBAC: Adding query filter params (type=network) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:10.034871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5440c4-025b-4796-89f9-c91ccc1565f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:10.037117 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5440c4-025b-4796-89f9-c91ccc1565f0 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:10.053091 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 386/767] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Jul 17 16:41:09 2022] GET /identity/v3/services?type=network => generated 350 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:10.054638 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5440c4-025b-4796-89f9-c91ccc1565f0 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:10.058178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-510aca96-312b-4366-b08f-2fbdce40c09d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:41:10.058178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:10.058178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:10.058178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:10.058178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:10.058178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:10.058178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:10.058178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:10.058178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:10.058178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:10.058178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:10.058178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:10.058178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:41:10.058178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:41:10.058178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:41:10.061301 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:41:10.061301 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:10.061301 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:10.061301 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:10.061301 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:10.061301 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:10.061301 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:10.061301 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:10.061301 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:10.061301 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:10.061301 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:10.061301 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:10.061301 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:10.061301 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:10.061301 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:10.061301 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:10.061301 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:10.063763 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:41:10.063763 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:41:10.063763 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:41:10.063763 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:41:10.063763 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:41:10.063763 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:41:10.063763 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:41:10.063763 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:10.074435 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 382/768] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:41:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:10.074435 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510aca96-312b-4366-b08f-2fbdce40c09d None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:10.088632 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:10.216131 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:10.246774 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5440c4-025b-4796-89f9-c91ccc1565f0 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:10.266782 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510aca96-312b-4366-b08f-2fbdce40c09d None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:10.313253 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34db4365-ec50-4f6e-9ac9-2d42ee4fa992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:10.313695 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a79e43-8704-4b93-942b-602d708bea31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:10.314240 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db4365-ec50-4f6e-9ac9-2d42ee4fa992 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:10.314358 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a79e43-8704-4b93-942b-602d708bea31 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:10.314517 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db4365-ec50-4f6e-9ac9-2d42ee4fa992 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:10.314618 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a79e43-8704-4b93-942b-602d708bea31 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:10.314833 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34db4365-ec50-4f6e-9ac9-2d42ee4fa992 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:10.314990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a79e43-8704-4b93-942b-602d708bea31 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:10.315699 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a79e43-8704-4b93-942b-602d708bea31 None admin] RBAC: Adding query filter params (interface=public, service_id=a55d2034057d47538a4cf304122b15c2, region_id=RegionOne) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:10.315995 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34db4365-ec50-4f6e-9ac9-2d42ee4fa992 None admin] RBAC: Adding query filter params (role.id=1025051975f64f689effa58dc4259e36, scope.project.id=927057f9a5af4833aeac9992203a223c, user.id=69482eeb1e8547aabca9f036b5c34f13) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:10.316136 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a79e43-8704-4b93-942b-602d708bea31 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:10.316516 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34db4365-ec50-4f6e-9ac9-2d42ee4fa992 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:10.317490 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a79e43-8704-4b93-942b-602d708bea31 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:10.319380 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34db4365-ec50-4f6e-9ac9-2d42ee4fa992 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:10.336747 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 387/769] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:41:10 2022] GET /identity/v3/role_assignments?user.id=69482eeb1e8547aabca9f036b5c34f13&scope.project.id=927057f9a5af4833aeac9992203a223c&role.id=1025051975f64f689effa58dc4259e36 => generated 259 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:10.337051 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 383/770] 10.222.0.40 () {56 vars in 1251 bytes} [Sun Jul 17 16:41:10 2022] GET /identity/v3/endpoints?service_id=a55d2034057d47538a4cf304122b15c2&interface=public®ion_id=RegionOne => generated 194 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:10.337216 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34db4365-ec50-4f6e-9ac9-2d42ee4fa992 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:10.337354 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f24c25-14dc-4f73-9ef0-fa9d240626bb None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:10.337354 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f24c25-14dc-4f73-9ef0-fa9d240626bb None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:10.337354 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f24c25-14dc-4f73-9ef0-fa9d240626bb None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:10.337354 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 384/771] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:10.337354 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f24c25-14dc-4f73-9ef0-fa9d240626bb None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:10.482754 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:10.558340 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34db4365-ec50-4f6e-9ac9-2d42ee4fa992 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:10.561944 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f24c25-14dc-4f73-9ef0-fa9d240626bb None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:10.608029 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428306c8-6bce-4395-a532-7fa9fe59d849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:10.609947 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428306c8-6bce-4395-a532-7fa9fe59d849 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:10.610345 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428306c8-6bce-4395-a532-7fa9fe59d849 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:10.610720 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428306c8-6bce-4395-a532-7fa9fe59d849 None admin] PATH_INFO: `/v3/projects` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:10.611774 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428306c8-6bce-4395-a532-7fa9fe59d849 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:10.612065 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428306c8-6bce-4395-a532-7fa9fe59d849 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:10.613684 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428306c8-6bce-4395-a532-7fa9fe59d849 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:10.616684 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:10.635701 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46205e3-6735-41c8-b481-bbda8e19de70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:10.635701 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46205e3-6735-41c8-b481-bbda8e19de70 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:10.635701 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46205e3-6735-41c8-b481-bbda8e19de70 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:10.635701 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46205e3-6735-41c8-b481-bbda8e19de70 None admin] PATH_INFO: `/v3/services` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:10.636126 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46205e3-6735-41c8-b481-bbda8e19de70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:10.636426 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 388/772] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:41:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:10.636740 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46205e3-6735-41c8-b481-bbda8e19de70 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:10.638365 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda8808-5c96-4fcc-9512-466ba91e8ea1 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:10.638893 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda8808-5c96-4fcc-9512-466ba91e8ea1 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:10.639382 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda8808-5c96-4fcc-9512-466ba91e8ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:10.648040 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 385/773] 10.222.0.40 () {56 vars in 1087 bytes} [Sun Jul 17 16:41:10 2022] GET /identity/v3/services => generated 772 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:10.649661 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46205e3-6735-41c8-b481-bbda8e19de70 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:10.672866 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-5bda8808-5c96-4fcc-9512-466ba91e8ea1 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:10.674588 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-5bda8808-5c96-4fcc-9512-466ba91e8ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:10.838638 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 389/774] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:41:10.889347 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bda8808-5c96-4fcc-9512-466ba91e8ea1 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:10.910586 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46205e3-6735-41c8-b481-bbda8e19de70 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:11.001681 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a4faf-3f4a-40ae-91af-0125301023f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:11.002797 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a4faf-3f4a-40ae-91af-0125301023f8 None admin] REQUEST_METHOD: `PUT` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:11.003301 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a4faf-3f4a-40ae-91af-0125301023f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:11.003683 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a4faf-3f4a-40ae-91af-0125301023f8 None admin] PATH_INFO: `/v3/projects/927057f9a5af4833aeac9992203a223c/users/03244087f1a34bddb11ea46aba11a148/roles/1025051975f64f689effa58dc4259e36` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:11.015292 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a4faf-3f4a-40ae-91af-0125301023f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=927057f9a5af4833aeac9992203a223c, user_id=03244087f1a34bddb11ea46aba11a148, role_id=1025051975f64f689effa58dc4259e36)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:11.018069 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a4faf-3f4a-40ae-91af-0125301023f8 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:11.048454 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:11.120291 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 386/775] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 17 16:41:10 2022] PUT /identity/v3/projects/927057f9a5af4833aeac9992203a223c/users/03244087f1a34bddb11ea46aba11a148/roles/1025051975f64f689effa58dc4259e36 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 17 16:41:11.132010 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bda8808-5c96-4fcc-9512-466ba91e8ea1 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:11.232084 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ee690-a180-4892-a6f7-2c24e40e75ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:11.233565 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ee690-a180-4892-a6f7-2c24e40e75ea None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:11.234086 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ee690-a180-4892-a6f7-2c24e40e75ea None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:11.234592 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ee690-a180-4892-a6f7-2c24e40e75ea None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:11.235887 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3ee690-a180-4892-a6f7-2c24e40e75ea None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:11.238394 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3ee690-a180-4892-a6f7-2c24e40e75ea None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:11.261940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-ad3ee690-a180-4892-a6f7-2c24e40e75ea None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 17 16:41:11.261940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:11.261940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 17 16:41:11.261940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 17 16:41:11.261940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 17 16:41:11.261940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 17 16:41:11.261940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 17 16:41:11.261940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:11.261940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 17 16:41:11.261940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:11.261940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 17 16:41:11.261940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:11.261940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:11.261940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:11.261940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:11.261940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:11.264221 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:11.264221 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:11.264221 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:11.264221 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:11.264221 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:11.264221 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:11.264221 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:11.264221 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 17 16:41:11.264221 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 17 16:41:11.264221 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 17 16:41:11.264221 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 17 16:41:11.264221 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:11.264221 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:11.264221 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:11.264221 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:11.264221 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:11.267415 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:11.267415 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:11.267415 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:11.267415 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:11.267415 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:11.267415 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:11.267415 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:11.267415 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:11.267415 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:11.267415 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:11.267415 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:11.267415 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 17 16:41:11.267415 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:11.267415 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 17 16:41:11.267415 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:11.267415 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 390/776] 10.222.0.40 () {56 vars in 1129 bytes} [Sun Jul 17 16:41:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:11.272220 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a4faf-3f4a-40ae-91af-0125301023f8 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:11.376039 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:11.462047 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a4faf-3f4a-40ae-91af-0125301023f8 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:11.520061 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9406114d-9ec8-48c8-a21c-8706d3336f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:11.520760 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406114d-9ec8-48c8-a21c-8706d3336f67 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:11.521067 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406114d-9ec8-48c8-a21c-8706d3336f67 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:11.521347 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406114d-9ec8-48c8-a21c-8706d3336f67 None admin] PATH_INFO: `/v3/services` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:11.522095 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9406114d-9ec8-48c8-a21c-8706d3336f67 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:11.522404 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9406114d-9ec8-48c8-a21c-8706d3336f67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:11.523920 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9406114d-9ec8-48c8-a21c-8706d3336f67 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:11.529754 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 387/777] 10.222.0.40 () {56 vars in 1124 bytes} [Sun Jul 17 16:41:11 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:11.538726 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ee690-a180-4892-a6f7-2c24e40e75ea None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:11.726016 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ee690-a180-4892-a6f7-2c24e40e75ea None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:11.768049 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:11.777913 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f8012-5cc8-4217-971a-25878724cec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:11.778607 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f8012-5cc8-4217-971a-25878724cec1 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:11.778898 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f8012-5cc8-4217-971a-25878724cec1 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:11.779180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f8012-5cc8-4217-971a-25878724cec1 None admin] PATH_INFO: `/v3/services` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:11.779920 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6f8012-5cc8-4217-971a-25878724cec1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:11.780870 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6f8012-5cc8-4217-971a-25878724cec1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:11.782297 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6f8012-5cc8-4217-971a-25878724cec1 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:11.793948 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 391/778] 10.222.0.40 () {56 vars in 1124 bytes} [Sun Jul 17 16:41:11 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:11.800312 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9406114d-9ec8-48c8-a21c-8706d3336f67 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:11.965802 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9406114d-9ec8-48c8-a21c-8706d3336f67 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:11.989546 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:12.013566 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfff9c-282d-461f-9b89-d92f9e908775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:12.014306 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfff9c-282d-461f-9b89-d92f9e908775 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:12.015552 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfff9c-282d-461f-9b89-d92f9e908775 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:12.015836 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfff9c-282d-461f-9b89-d92f9e908775 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:12.016654 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfff9c-282d-461f-9b89-d92f9e908775 None admin] RBAC: Adding query filter params (interface=public, service_id=6d9f83719457495eb49cd4b88beba609, region_id=RegionOne) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:12.016996 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfff9c-282d-461f-9b89-d92f9e908775 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:12.019410 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfff9c-282d-461f-9b89-d92f9e908775 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:12.029863 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 388/779] 10.222.0.40 () {56 vars in 1251 bytes} [Sun Jul 17 16:41:11 2022] GET /identity/v3/endpoints?service_id=6d9f83719457495eb49cd4b88beba609&interface=public®ion_id=RegionOne => generated 194 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:12.035720 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f8012-5cc8-4217-971a-25878724cec1 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:12.243711 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f8012-5cc8-4217-971a-25878724cec1 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:12.328599 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662b5d6-0bc2-4617-a7e0-ca08903c1d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:12.329283 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662b5d6-0bc2-4617-a7e0-ca08903c1d9b None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:12.329570 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662b5d6-0bc2-4617-a7e0-ca08903c1d9b None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:12.329850 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662b5d6-0bc2-4617-a7e0-ca08903c1d9b None admin] PATH_INFO: `/v3/services` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:12.330556 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2662b5d6-0bc2-4617-a7e0-ca08903c1d9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:12.332010 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2662b5d6-0bc2-4617-a7e0-ca08903c1d9b None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:12.342132 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 392/780] 10.222.0.40 () {56 vars in 1087 bytes} [Sun Jul 17 16:41:12 2022] GET /identity/v3/services => generated 772 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:12.362233 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1257e7-a045-4499-aa79-51321a2a78e7 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:12.362571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1257e7-a045-4499-aa79-51321a2a78e7 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:12.362804 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1257e7-a045-4499-aa79-51321a2a78e7 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:12.364424 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 389/781] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:12.379818 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61087b0d-762b-49cf-8b53-2342d6fb11bd None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:12.380124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61087b0d-762b-49cf-8b53-2342d6fb11bd None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:12.380367 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61087b0d-762b-49cf-8b53-2342d6fb11bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:12.404732 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-61087b0d-762b-49cf-8b53-2342d6fb11bd None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:12.407239 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-61087b0d-762b-49cf-8b53-2342d6fb11bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:12.506069 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:12.530713 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 393/782] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:41:12.565834 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1257e7-a045-4499-aa79-51321a2a78e7 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:12.693315 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1257e7-a045-4499-aa79-51321a2a78e7 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:12.741934 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab23f1-a3e0-44e4-a5e9-48fac761d930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:12.742770 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab23f1-a3e0-44e4-a5e9-48fac761d930 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:12.743065 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab23f1-a3e0-44e4-a5e9-48fac761d930 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:12.743396 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab23f1-a3e0-44e4-a5e9-48fac761d930 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:12.758097 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab23f1-a3e0-44e4-a5e9-48fac761d930 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:41:12.766536 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-60ab23f1-a3e0-44e4-a5e9-48fac761d930 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:41:12.766536 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:12.766536 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:12.766536 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:12.766536 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:12.766536 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:12.766536 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:12.766536 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:12.766536 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:12.766536 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:12.766536 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:12.766536 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:12.766536 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:41:12.766536 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:41:12.766536 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:41:12.766536 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:41:12.768414 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:41:12.768414 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:41:12.768414 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:41:12.768414 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:41:12.768414 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:12.768414 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:12.768414 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:12.768414 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:12.768414 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:12.768414 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:12.768414 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:12.768414 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:12.768414 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:12.768414 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:12.768414 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:12.768414 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:12.770139 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:12.770139 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:12.770139 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:12.770139 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:12.770139 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:41:12.770139 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:41:12.770139 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:41:12.770139 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:41:12.770139 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:41:12.770139 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:41:12.770139 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:41:12.770139 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:12.770139 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 390/783] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 17 16:41:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:12.777986 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61087b0d-762b-49cf-8b53-2342d6fb11bd None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:12.917902 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61087b0d-762b-49cf-8b53-2342d6fb11bd None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:12.958276 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c3a45-d8b9-4232-acd0-2b31e5fa3616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:12.958974 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c3a45-d8b9-4232-acd0-2b31e5fa3616 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:12.959255 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c3a45-d8b9-4232-acd0-2b31e5fa3616 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:12.959520 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c3a45-d8b9-4232-acd0-2b31e5fa3616 None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:12.960279 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c3a45-d8b9-4232-acd0-2b31e5fa3616 None admin] RBAC: Adding query filter params (name=service) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:12.960582 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c3a45-d8b9-4232-acd0-2b31e5fa3616 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:12.962031 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c3a45-d8b9-4232-acd0-2b31e5fa3616 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:12.972614 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 394/784] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 17 16:41:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:12.979070 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab23f1-a3e0-44e4-a5e9-48fac761d930 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:13.098669 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:13.248353 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab23f1-a3e0-44e4-a5e9-48fac761d930 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:13.315583 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03880f1b-70b2-47c6-b24a-e01cab3dbda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:13.316415 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03880f1b-70b2-47c6-b24a-e01cab3dbda1 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:13.316704 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03880f1b-70b2-47c6-b24a-e01cab3dbda1 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:13.317010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03880f1b-70b2-47c6-b24a-e01cab3dbda1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:13.333682 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03880f1b-70b2-47c6-b24a-e01cab3dbda1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:13.335196 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03880f1b-70b2-47c6-b24a-e01cab3dbda1 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:13.355223 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-03880f1b-70b2-47c6-b24a-e01cab3dbda1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:13.355223 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:13.355223 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:13.355223 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:13.355223 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:13.355223 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:13.355223 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:13.355223 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:13.355223 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:13.355223 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:13.355223 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:13.355223 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:13.355223 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:41:13.355223 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:41:13.355223 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:41:13.356955 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:41:13.356955 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:13.356955 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:13.356955 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:13.356955 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:13.356955 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:13.356955 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:13.356955 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:13.356955 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:13.356955 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:13.356955 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:13.356955 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:13.356955 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:13.356955 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:13.356955 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:13.356955 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:13.356955 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:13.359382 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:41:13.359382 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:41:13.359382 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:13.359382 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:13.359382 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 391/785] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:41:13.366224 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c3a45-d8b9-4232-acd0-2b31e5fa3616 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:13.370366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09794050-67ac-47d5-8cf3-52897e1becf6 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:13.370920 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09794050-67ac-47d5-8cf3-52897e1becf6 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:13.371213 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09794050-67ac-47d5-8cf3-52897e1becf6 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:13.373061 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 392/786] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:13.379484 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57feeb-dd6d-478f-9287-6ed1c9dc3fcf None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:13.379828 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57feeb-dd6d-478f-9287-6ed1c9dc3fcf None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:13.380110 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57feeb-dd6d-478f-9287-6ed1c9dc3fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:13.402459 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-cc57feeb-dd6d-478f-9287-6ed1c9dc3fcf None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:13.403745 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-cc57feeb-dd6d-478f-9287-6ed1c9dc3fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:13.455983 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:13.570592 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 393/787] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:41:13.576099 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c3a45-d8b9-4232-acd0-2b31e5fa3616 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:13.627087 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57feeb-dd6d-478f-9287-6ed1c9dc3fcf None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:13.646776 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefcb2b1-26d6-4011-b4d2-9024ee65e499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:13.647535 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefcb2b1-26d6-4011-b4d2-9024ee65e499 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:13.647827 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefcb2b1-26d6-4011-b4d2-9024ee65e499 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:13.648109 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefcb2b1-26d6-4011-b4d2-9024ee65e499 None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:13.648882 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefcb2b1-26d6-4011-b4d2-9024ee65e499 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:13.649198 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefcb2b1-26d6-4011-b4d2-9024ee65e499 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:13.650610 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefcb2b1-26d6-4011-b4d2-9024ee65e499 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:13.680179 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 395/788] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:41:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:13.681803 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:13.685989 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefcb2b1-26d6-4011-b4d2-9024ee65e499 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:13.785591 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57feeb-dd6d-478f-9287-6ed1c9dc3fcf None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:13.825929 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e26e5-18b3-4ac9-badc-42f87faa18bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:13.827392 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e26e5-18b3-4ac9-badc-42f87faa18bd None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:13.827733 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e26e5-18b3-4ac9-badc-42f87faa18bd None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:13.828015 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e26e5-18b3-4ac9-badc-42f87faa18bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:13.841671 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797e26e5-18b3-4ac9-badc-42f87faa18bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:41:13.850641 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-797e26e5-18b3-4ac9-badc-42f87faa18bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:41:13.850641 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:13.850641 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:13.850641 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:13.850641 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:13.850641 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:13.850641 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:13.850641 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:13.850641 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:13.850641 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:13.850641 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:13.850641 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:13.850641 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:41:13.850641 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:41:13.850641 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:41:13.850641 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:41:13.851657 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:41:13.851657 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:41:13.851657 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:41:13.851657 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:41:13.851657 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:13.851657 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:13.851657 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:13.851657 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:13.851657 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:13.851657 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:13.851657 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:13.851657 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:13.851657 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:13.851657 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:13.851657 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:13.851657 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:13.852493 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:13.852493 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:13.852493 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:13.852493 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:13.852493 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:41:13.852493 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:41:13.852493 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:41:13.852493 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:41:13.852493 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:41:13.852493 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:41:13.852493 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:41:13.852493 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:13.857022 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 394/789] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:41:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:41:13.858969 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e26e5-18b3-4ac9-badc-42f87faa18bd None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:13.964558 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:13.988402 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e26e5-18b3-4ac9-badc-42f87faa18bd None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:14.024777 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefcb2b1-26d6-4011-b4d2-9024ee65e499 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:14.031945 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d707acbf-e229-47f2-94b8-82704e253a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:14.032653 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707acbf-e229-47f2-94b8-82704e253a64 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:14.032981 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707acbf-e229-47f2-94b8-82704e253a64 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:14.033266 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707acbf-e229-47f2-94b8-82704e253a64 None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:14.034044 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d707acbf-e229-47f2-94b8-82704e253a64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:14.034391 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d707acbf-e229-47f2-94b8-82704e253a64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:14.035810 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d707acbf-e229-47f2-94b8-82704e253a64 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:14.040958 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:14.051689 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 395/790] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:41:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:41:14.053190 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d707acbf-e229-47f2-94b8-82704e253a64 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:14.105795 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae4060-245a-4bdd-8477-78f61c5d5dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:14.106633 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae4060-245a-4bdd-8477-78f61c5d5dad None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:14.106918 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae4060-245a-4bdd-8477-78f61c5d5dad None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:14.107196 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae4060-245a-4bdd-8477-78f61c5d5dad None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:14.115741 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae4060-245a-4bdd-8477-78f61c5d5dad None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:14.117247 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae4060-245a-4bdd-8477-78f61c5d5dad None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:14.134575 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-52ae4060-245a-4bdd-8477-78f61c5d5dad None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 17 16:41:14.134575 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:14.134575 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:14.134575 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:14.134575 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:14.134575 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:14.134575 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:14.134575 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:14.134575 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:14.134575 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:14.134575 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:14.134575 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:14.134575 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:41:14.134575 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:41:14.134575 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:41:14.134575 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:41:14.135631 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:14.135631 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:14.135631 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:41:14.135631 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:41:14.135631 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:41:14.135631 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:41:14.135631 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:14.135631 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:14.135631 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:14.135631 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:14.135631 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:14.135631 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:14.135631 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:14.135631 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:14.135631 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:14.135631 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:14.135631 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:14.135631 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:14.136587 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:14.136587 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:14.136587 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:41:14.136587 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:41:14.136587 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:41:14.136587 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:41:14.136587 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:41:14.136587 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:41:14.136587 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 17 16:41:14.136587 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:14.141333 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 396/791] 10.222.0.40 () {56 vars in 1117 bytes} [Sun Jul 17 16:41:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 450 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:14.142885 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23896fc8-b060-43fc-86b2-9291c63eee47 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:14.143170 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23896fc8-b060-43fc-86b2-9291c63eee47 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:14.143443 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23896fc8-b060-43fc-86b2-9291c63eee47 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:14.145131 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 397/792] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:14.145926 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23896fc8-b060-43fc-86b2-9291c63eee47 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:14.175595 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d707acbf-e229-47f2-94b8-82704e253a64 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:14.218623 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee631ef-d656-476a-b70e-bcafad2d9efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:14.219341 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee631ef-d656-476a-b70e-bcafad2d9efb None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:14.219632 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee631ef-d656-476a-b70e-bcafad2d9efb None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:14.219917 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee631ef-d656-476a-b70e-bcafad2d9efb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:14.238344 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee631ef-d656-476a-b70e-bcafad2d9efb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:14.240311 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee631ef-d656-476a-b70e-bcafad2d9efb None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:14.243827 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:14.262903 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-5ee631ef-d656-476a-b70e-bcafad2d9efb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:14.262903 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:14.262903 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:14.262903 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:14.262903 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:14.262903 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:14.262903 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:14.262903 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:14.262903 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:14.262903 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:14.262903 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:14.262903 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:14.262903 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:41:14.262903 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:41:14.262903 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:41:14.263861 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:41:14.263861 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:14.263861 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:14.263861 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:14.263861 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:14.263861 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:14.263861 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:14.263861 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:14.263861 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:14.263861 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:14.263861 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:14.263861 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:14.263861 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:14.263861 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:14.263861 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:14.263861 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:14.263861 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:14.265888 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:41:14.265888 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:41:14.265888 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:14.265888 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:14.266118 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23896fc8-b060-43fc-86b2-9291c63eee47 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:14.268246 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 396/793] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:41:14.270061 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c513635-51a5-4585-b6dd-4649407a2e34 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:14.270592 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c513635-51a5-4585-b6dd-4649407a2e34 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:14.271075 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c513635-51a5-4585-b6dd-4649407a2e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:14.291439 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-1c513635-51a5-4585-b6dd-4649407a2e34 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:14.293289 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-1c513635-51a5-4585-b6dd-4649407a2e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:14.304305 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a8d9c-6456-42c1-b07e-8d1d53eda560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:14.305025 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a8d9c-6456-42c1-b07e-8d1d53eda560 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:14.305310 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a8d9c-6456-42c1-b07e-8d1d53eda560 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:14.305583 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a8d9c-6456-42c1-b07e-8d1d53eda560 None admin] PATH_INFO: `/v3/users` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:14.306517 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893a8d9c-6456-42c1-b07e-8d1d53eda560 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:14.306828 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893a8d9c-6456-42c1-b07e-8d1d53eda560 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:14.308285 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893a8d9c-6456-42c1-b07e-8d1d53eda560 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:14.334504 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 398/794] 10.222.0.40 () {56 vars in 1149 bytes} [Sun Jul 17 16:41:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:14.335584 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a8d9c-6456-42c1-b07e-8d1d53eda560 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:14.363704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:14.379178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 397/795] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:41:14.380393 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c513635-51a5-4585-b6dd-4649407a2e34 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:14.449750 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a8d9c-6456-42c1-b07e-8d1d53eda560 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:14.500870 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c513635-51a5-4585-b6dd-4649407a2e34 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:14.502345 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e35b9f4-d2d0-4f12-8917-81ab8ea0eb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:14.503295 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35b9f4-d2d0-4f12-8917-81ab8ea0eb53 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:14.504308 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35b9f4-d2d0-4f12-8917-81ab8ea0eb53 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:14.504308 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35b9f4-d2d0-4f12-8917-81ab8ea0eb53 None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:14.505745 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e35b9f4-d2d0-4f12-8917-81ab8ea0eb53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:14.506300 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e35b9f4-d2d0-4f12-8917-81ab8ea0eb53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:14.508993 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e35b9f4-d2d0-4f12-8917-81ab8ea0eb53 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:14.535458 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 399/796] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:41:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:14.535458 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d8e39-1a5b-44da-9f6f-b45e16b9bd0e None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:14.535458 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d8e39-1a5b-44da-9f6f-b45e16b9bd0e None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:14.535458 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d8e39-1a5b-44da-9f6f-b45e16b9bd0e None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:14.536730 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 400/797] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:14.538013 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:14.544472 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d8e39-1a5b-44da-9f6f-b45e16b9bd0e None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:14.547329 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd106f8-fba2-4880-a6f1-7b5d07bd4bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:14.549746 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd106f8-fba2-4880-a6f1-7b5d07bd4bad None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:14.549746 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd106f8-fba2-4880-a6f1-7b5d07bd4bad None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:14.549746 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd106f8-fba2-4880-a6f1-7b5d07bd4bad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:14.560539 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd106f8-fba2-4880-a6f1-7b5d07bd4bad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:14.563038 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd106f8-fba2-4880-a6f1-7b5d07bd4bad None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:14.583917 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-5fd106f8-fba2-4880-a6f1-7b5d07bd4bad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:14.583917 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:14.583917 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:14.583917 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:14.583917 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:14.583917 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:14.583917 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:14.583917 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:14.583917 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:14.583917 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:14.583917 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:14.583917 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:14.583917 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:41:14.583917 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:41:14.583917 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:41:14.584901 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:41:14.584901 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:14.584901 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:14.584901 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:14.584901 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:14.584901 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:14.584901 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:14.584901 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:14.584901 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:14.584901 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:14.584901 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:14.584901 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:14.584901 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:14.584901 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:14.584901 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:14.584901 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:14.584901 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:14.585790 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:41:14.585790 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:41:14.585790 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:14.585790 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:14.593497 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 398/798] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:14.593497 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd106f8-fba2-4880-a6f1-7b5d07bd4bad None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:14.703458 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:14.730391 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd106f8-fba2-4880-a6f1-7b5d07bd4bad None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:14.731685 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d8e39-1a5b-44da-9f6f-b45e16b9bd0e None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:14.781713 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e9f6ff-a7d7-4d99-a957-5207e84ed9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:14.782527 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9f6ff-a7d7-4d99-a957-5207e84ed9bd None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:14.782813 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9f6ff-a7d7-4d99-a957-5207e84ed9bd None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:14.783158 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e9f6ff-a7d7-4d99-a957-5207e84ed9bd None admin] PATH_INFO: `/v3/users/swift` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:14.784114 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315c9df-a3e5-4871-9815-87e5da20d689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:14.785149 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315c9df-a3e5-4871-9815-87e5da20d689 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:14.785636 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315c9df-a3e5-4871-9815-87e5da20d689 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:14.786113 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315c9df-a3e5-4871-9815-87e5da20d689 None admin] PATH_INFO: `/v3/services/network` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:14.787361 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315c9df-a3e5-4871-9815-87e5da20d689 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:14.789243 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315c9df-a3e5-4871-9815-87e5da20d689 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:14.793037 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e9f6ff-a7d7-4d99-a957-5207e84ed9bd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:14.794504 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e9f6ff-a7d7-4d99-a957-5207e84ed9bd None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:14.803894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-6315c9df-a3e5-4871-9815-87e5da20d689 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 17 16:41:14.803894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:14.803894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 17 16:41:14.803894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 17 16:41:14.803894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 17 16:41:14.803894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 17 16:41:14.803894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 17 16:41:14.803894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:14.803894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 17 16:41:14.803894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:14.803894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 17 16:41:14.803894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:14.803894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:14.803894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:14.803894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:14.803894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:14.803894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:14.805882 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:14.805882 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:14.805882 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:14.805882 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:14.805882 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:14.805882 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:14.805882 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 17 16:41:14.805882 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 17 16:41:14.805882 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 17 16:41:14.805882 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 17 16:41:14.805882 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:14.805882 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:14.805882 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:14.805882 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:14.805882 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:14.805882 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:14.815546 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:14.815546 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:14.815546 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:14.815546 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:14.815546 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:14.815546 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:14.815546 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:14.815546 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:14.815546 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:14.815546 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:14.815546 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 17 16:41:14.815546 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:14.815546 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 17 16:41:14.815546 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:14.815546 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 401/799] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:41:14 2022] GET /identity/v3/services/network => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:14.819028 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4fd999-8791-4435-86d3-56a84dc9140e None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:14.819028 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4fd999-8791-4435-86d3-56a84dc9140e None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:14.819028 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4fd999-8791-4435-86d3-56a84dc9140e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:14.819626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-99e9f6ff-a7d7-4d99-a957-5207e84ed9bd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 17 16:41:14.819626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:14.819626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:14.819626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:14.819626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:14.819626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:14.819626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:14.819626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:14.819626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:14.819626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:14.819626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:14.819626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:14.819626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:41:14.819626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:41:14.819626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:41:14.819626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:41:14.827224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:14.827224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:14.827224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:41:14.827224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:41:14.827224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:41:14.827224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:41:14.827224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:14.827224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:14.827224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:14.827224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:14.827224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:14.827224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:14.827224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:14.827224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:14.827224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:14.827224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:14.827224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:14.827224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:14.830465 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:14.830465 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:14.830465 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:41:14.830465 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:41:14.830465 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:41:14.830465 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:41:14.830465 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:41:14.830465 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:41:14.830465 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 17 16:41:14.830465 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:14.830465 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 399/800] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:41:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:14.830465 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e9f6ff-a7d7-4d99-a957-5207e84ed9bd None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:14.841650 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-4c4fd999-8791-4435-86d3-56a84dc9140e None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:14.844048 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-4c4fd999-8791-4435-86d3-56a84dc9140e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:14.873543 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:14.926683 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 402/801] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:41:14.927686 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4fd999-8791-4435-86d3-56a84dc9140e None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:14.971238 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e9f6ff-a7d7-4d99-a957-5207e84ed9bd None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:15.013510 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d39e33-066e-45f6-a7e1-6725e0c389f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:15.014288 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d39e33-066e-45f6-a7e1-6725e0c389f1 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:15.015144 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d39e33-066e-45f6-a7e1-6725e0c389f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:15.015666 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d39e33-066e-45f6-a7e1-6725e0c389f1 None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:15.016570 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d39e33-066e-45f6-a7e1-6725e0c389f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:15.016915 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d39e33-066e-45f6-a7e1-6725e0c389f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:15.018325 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d39e33-066e-45f6-a7e1-6725e0c389f1 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:15.026980 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:15.032916 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 400/802] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:41:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:15.038673 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d39e33-066e-45f6-a7e1-6725e0c389f1 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:15.052718 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4fd999-8791-4435-86d3-56a84dc9140e None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:15.096127 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38826b5a-f717-4df1-906c-c8328e2268ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:15.097267 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38826b5a-f717-4df1-906c-c8328e2268ae None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:15.097554 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38826b5a-f717-4df1-906c-c8328e2268ae None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:15.097835 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38826b5a-f717-4df1-906c-c8328e2268ae None admin] PATH_INFO: `/v3/users` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:15.098691 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38826b5a-f717-4df1-906c-c8328e2268ae None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:15.099009 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38826b5a-f717-4df1-906c-c8328e2268ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:15.100458 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38826b5a-f717-4df1-906c-c8328e2268ae None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:15.120692 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 403/803] 10.222.0.40 () {56 vars in 1135 bytes} [Sun Jul 17 16:41:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:15.121609 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38826b5a-f717-4df1-906c-c8328e2268ae None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:15.188636 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:15.192753 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d39e33-066e-45f6-a7e1-6725e0c389f1 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:15.231329 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6b909-073e-446e-a9b4-94be8165b560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:15.232352 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6b909-073e-446e-a9b4-94be8165b560 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:15.233141 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6b909-073e-446e-a9b4-94be8165b560 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:15.234841 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6b909-073e-446e-a9b4-94be8165b560 None admin] PATH_INFO: `/v3/services` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:15.235719 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a6b909-073e-446e-a9b4-94be8165b560 None admin] RBAC: Adding query filter params (name=network) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:15.236139 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a6b909-073e-446e-a9b4-94be8165b560 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:15.237820 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a6b909-073e-446e-a9b4-94be8165b560 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:15.243903 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 401/804] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Jul 17 16:41:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:15.244752 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6b909-073e-446e-a9b4-94be8165b560 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:15.266591 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38826b5a-f717-4df1-906c-c8328e2268ae None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:15.309048 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59424976-4686-4379-85b1-a8aba6f13c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:15.310250 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59424976-4686-4379-85b1-a8aba6f13c8a None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:15.310696 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59424976-4686-4379-85b1-a8aba6f13c8a None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:15.311862 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59424976-4686-4379-85b1-a8aba6f13c8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:15.316153 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:15.331402 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59424976-4686-4379-85b1-a8aba6f13c8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:41:15.345457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-59424976-4686-4379-85b1-a8aba6f13c8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:41:15.345457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:15.345457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:15.345457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:15.345457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:15.345457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:15.345457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:15.345457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:15.345457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:15.345457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:15.345457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:15.345457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:15.345457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:41:15.345457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:41:15.345457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:41:15.345457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:41:15.347399 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:41:15.347399 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:41:15.347399 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:41:15.347399 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:41:15.347399 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:15.347399 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:15.347399 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:15.347399 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:15.347399 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:15.347399 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:15.347399 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:15.347399 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:15.347399 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:15.347399 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:15.347399 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:15.347399 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:15.349538 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:15.349538 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:15.349538 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:15.349538 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:15.349538 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:41:15.349538 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:41:15.349538 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:41:15.349538 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:41:15.349538 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:41:15.349538 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:41:15.349538 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:41:15.349538 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:15.349538 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 404/805] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:41:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:15.351058 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b977b82-b59b-4c49-be4d-b7021cc32013 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:15.351058 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b977b82-b59b-4c49-be4d-b7021cc32013 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:15.351058 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b977b82-b59b-4c49-be4d-b7021cc32013 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:15.352753 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 405/806] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:15.354280 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b977b82-b59b-4c49-be4d-b7021cc32013 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:15.399147 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6b909-073e-446e-a9b4-94be8165b560 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:15.453698 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b1a64-ed1d-434b-beb1-778ac3cc9b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:15.455341 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b1a64-ed1d-434b-beb1-778ac3cc9b4d None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:15.456302 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b1a64-ed1d-434b-beb1-778ac3cc9b4d None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:15.456835 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b1a64-ed1d-434b-beb1-778ac3cc9b4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:15.470362 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b1a64-ed1d-434b-beb1-778ac3cc9b4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:15.472929 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b1a64-ed1d-434b-beb1-778ac3cc9b4d None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:15.485674 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b977b82-b59b-4c49-be4d-b7021cc32013 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:15.497409 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-579b1a64-ed1d-434b-beb1-778ac3cc9b4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:41:15.497409 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:15.497409 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:15.497409 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:15.497409 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:15.497409 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:15.497409 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:15.497409 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:15.497409 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:15.497409 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:15.497409 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:15.497409 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:15.497409 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:41:15.497409 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:41:15.497409 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:41:15.498468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:41:15.498468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:15.498468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:15.498468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:15.498468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:15.498468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:15.498468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:15.498468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:15.498468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:15.498468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:15.498468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:15.498468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:15.498468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:15.498468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:15.498468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:15.498468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:15.498468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:15.500017 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:41:15.500017 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:41:15.500017 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:41:15.500017 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:41:15.500017 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:41:15.500017 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:41:15.500017 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:41:15.500017 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:15.500461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 402/807] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:41:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:15.501704 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b1a64-ed1d-434b-beb1-778ac3cc9b4d None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:15.533055 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddfabab-eada-4e4e-ac0a-6295c394439c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:15.533857 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfabab-eada-4e4e-ac0a-6295c394439c None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:15.534142 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfabab-eada-4e4e-ac0a-6295c394439c None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:15.534436 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddfabab-eada-4e4e-ac0a-6295c394439c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:15.534592 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:15.547477 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddfabab-eada-4e4e-ac0a-6295c394439c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:15.549051 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddfabab-eada-4e4e-ac0a-6295c394439c None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:15.569745 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-0ddfabab-eada-4e4e-ac0a-6295c394439c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:15.569745 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:15.569745 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:15.569745 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:15.569745 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:15.569745 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:15.569745 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:15.569745 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:15.569745 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:15.569745 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:15.569745 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:15.569745 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:15.569745 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:41:15.569745 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:41:15.569745 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:41:15.571569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:41:15.571569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:15.571569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:15.571569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:15.571569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:15.571569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:15.571569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:15.571569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:15.571569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:15.571569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:15.571569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:15.571569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:15.571569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:15.571569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:15.571569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:15.571569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:15.571569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:15.573428 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:41:15.573428 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:41:15.573428 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:15.573428 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:15.573428 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 406/808] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:15.573428 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddfabab-eada-4e4e-ac0a-6295c394439c None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:15.636680 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b1a64-ed1d-434b-beb1-778ac3cc9b4d None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:15.644919 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:15.677280 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e9706-0337-4c5a-a51b-c1785e9bd94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:15.677959 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e9706-0337-4c5a-a51b-c1785e9bd94b None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:15.678234 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e9706-0337-4c5a-a51b-c1785e9bd94b None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:15.679042 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e9706-0337-4c5a-a51b-c1785e9bd94b None admin] PATH_INFO: `/v3/services` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:15.679292 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e9706-0337-4c5a-a51b-c1785e9bd94b None admin] RBAC: Adding query filter params (type=network) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:15.679552 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e9706-0337-4c5a-a51b-c1785e9bd94b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:15.680958 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e9706-0337-4c5a-a51b-c1785e9bd94b None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:15.686618 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 403/809] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Jul 17 16:41:15 2022] GET /identity/v3/services?type=network => generated 350 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:15.688257 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271fc611-8b9d-4e4d-bd7c-4dcd744e39e3 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:15.688549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271fc611-8b9d-4e4d-bd7c-4dcd744e39e3 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:15.688851 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271fc611-8b9d-4e4d-bd7c-4dcd744e39e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:15.707717 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-271fc611-8b9d-4e4d-bd7c-4dcd744e39e3 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:15.709289 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-271fc611-8b9d-4e4d-bd7c-4dcd744e39e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:15.735145 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddfabab-eada-4e4e-ac0a-6295c394439c None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:15.774827 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28416c1-248c-4d84-8653-3f342bff36a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:15.775492 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28416c1-248c-4d84-8653-3f342bff36a4 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:15.775789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28416c1-248c-4d84-8653-3f342bff36a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:15.776060 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28416c1-248c-4d84-8653-3f342bff36a4 None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:15.776838 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28416c1-248c-4d84-8653-3f342bff36a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:15.777150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28416c1-248c-4d84-8653-3f342bff36a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:15.778537 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28416c1-248c-4d84-8653-3f342bff36a4 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:15.788383 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 407/810] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:41:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:15.789266 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28416c1-248c-4d84-8653-3f342bff36a4 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:15.814859 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 404/811] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:41:15.825310 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271fc611-8b9d-4e4d-bd7c-4dcd744e39e3 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:15.861494 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:15.925575 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28416c1-248c-4d84-8653-3f342bff36a4 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:15.938892 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:15.973862 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53575c8a-808e-4df8-a10c-d72f9dba2cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:15.974932 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53575c8a-808e-4df8-a10c-d72f9dba2cbf None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:15.975123 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271fc611-8b9d-4e4d-bd7c-4dcd744e39e3 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:15.975739 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53575c8a-808e-4df8-a10c-d72f9dba2cbf None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:15.976496 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53575c8a-808e-4df8-a10c-d72f9dba2cbf None admin] PATH_INFO: `/v3/projects` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:15.977953 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53575c8a-808e-4df8-a10c-d72f9dba2cbf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:15.978795 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53575c8a-808e-4df8-a10c-d72f9dba2cbf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:15.980986 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53575c8a-808e-4df8-a10c-d72f9dba2cbf None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:16.008516 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 408/812] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:41:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:16.008516 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53575c8a-808e-4df8-a10c-d72f9dba2cbf None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:16.033137 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11b35b2-7d0a-41db-ba56-5841f060c3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:16.034186 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b35b2-7d0a-41db-ba56-5841f060c3d6 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:16.034649 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b35b2-7d0a-41db-ba56-5841f060c3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:16.035086 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b35b2-7d0a-41db-ba56-5841f060c3d6 None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:16.036305 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11b35b2-7d0a-41db-ba56-5841f060c3d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:16.036837 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11b35b2-7d0a-41db-ba56-5841f060c3d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:16.039219 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11b35b2-7d0a-41db-ba56-5841f060c3d6 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:16.057944 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 405/813] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:41:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:16.059365 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11b35b2-7d0a-41db-ba56-5841f060c3d6 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:16.164701 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53575c8a-808e-4df8-a10c-d72f9dba2cbf None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:16.190071 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:16.213493 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4f511-35e1-4312-a094-edc27b24dbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:16.214362 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4f511-35e1-4312-a094-edc27b24dbb7 None admin] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:16.214725 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4f511-35e1-4312-a094-edc27b24dbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:16.215079 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4f511-35e1-4312-a094-edc27b24dbb7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:16.216101 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb4f511-35e1-4312-a094-edc27b24dbb7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:16.219081 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb4f511-35e1-4312-a094-edc27b24dbb7 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:16.219287 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11b35b2-7d0a-41db-ba56-5841f060c3d6 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:16.260245 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e338d-9e9f-4a7a-ba78-acfc71e20957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:16.261096 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e338d-9e9f-4a7a-ba78-acfc71e20957 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:16.261381 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e338d-9e9f-4a7a-ba78-acfc71e20957 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:16.261657 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e338d-9e9f-4a7a-ba78-acfc71e20957 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:16.274185 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e338d-9e9f-4a7a-ba78-acfc71e20957 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:16.275648 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102e338d-9e9f-4a7a-ba78-acfc71e20957 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:16.295030 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-102e338d-9e9f-4a7a-ba78-acfc71e20957 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:16.295030 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:16.295030 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:16.295030 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:16.295030 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:16.295030 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:16.295030 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:16.295030 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:16.295030 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:16.295030 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:16.295030 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:16.295030 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:16.295030 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:41:16.295030 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:41:16.295030 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:41:16.297048 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:41:16.297048 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:16.297048 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:16.297048 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:16.297048 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:16.297048 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:16.297048 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:16.297048 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:16.297048 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:16.297048 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:16.297048 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:16.297048 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:16.297048 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:16.297048 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:16.297048 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:16.297048 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:16.297048 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:16.298157 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:41:16.298157 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:41:16.298157 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:16.298157 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:16.298157 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 406/814] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:16.298676 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e338d-9e9f-4a7a-ba78-acfc71e20957 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:16.323658 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 409/815] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 17 16:41:16 2022] POST /identity/v3/endpoints => generated 327 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:41:16.325616 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4f511-35e1-4312-a094-edc27b24dbb7 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:16.381275 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:16.479291 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e338d-9e9f-4a7a-ba78-acfc71e20957 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:16.519189 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:16.532743 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4f511-35e1-4312-a094-edc27b24dbb7 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:16.537757 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917dbefb-0211-4b07-afcc-f72ccf9ae2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:16.539001 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917dbefb-0211-4b07-afcc-f72ccf9ae2ac None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:16.539293 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917dbefb-0211-4b07-afcc-f72ccf9ae2ac None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:16.539686 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917dbefb-0211-4b07-afcc-f72ccf9ae2ac None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:16.540594 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917dbefb-0211-4b07-afcc-f72ccf9ae2ac None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:16.542750 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917dbefb-0211-4b07-afcc-f72ccf9ae2ac None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:16.564437 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-917dbefb-0211-4b07-afcc-f72ccf9ae2ac None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 17 16:41:16.564437 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:16.564437 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 17 16:41:16.564437 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 17 16:41:16.564437 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 17 16:41:16.564437 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 17 16:41:16.564437 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 17 16:41:16.564437 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:16.564437 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 17 16:41:16.564437 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:16.564437 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 17 16:41:16.564437 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:16.564437 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:16.564437 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:16.564437 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:16.564437 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:16.566526 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:16.566526 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:16.566526 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:16.566526 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:16.566526 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:16.566526 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:16.566526 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:16.566526 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 17 16:41:16.566526 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 17 16:41:16.566526 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 17 16:41:16.566526 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 17 16:41:16.566526 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:16.566526 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:16.566526 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:16.566526 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:16.566526 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:16.568443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:16.568443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:16.568443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:16.568443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:16.568443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:16.568443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:16.568443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:16.568443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:16.568443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:16.568443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:16.568443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:16.568443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 17 16:41:16.568443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:16.568443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 17 16:41:16.568443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:16.568443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 407/816] 10.222.0.40 () {56 vars in 1129 bytes} [Sun Jul 17 16:41:16 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:16.570341 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917dbefb-0211-4b07-afcc-f72ccf9ae2ac None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:16.607943 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa42d69-2946-4aac-9ed7-928b74c41167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:16.610020 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa42d69-2946-4aac-9ed7-928b74c41167 None admin] REQUEST_METHOD: `PUT` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:16.610478 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa42d69-2946-4aac-9ed7-928b74c41167 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:16.610650 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa42d69-2946-4aac-9ed7-928b74c41167 None admin] PATH_INFO: `/v3/projects/927057f9a5af4833aeac9992203a223c/users/37ed5f72dc8c4a5685bd1245cadcdf7e/roles/adb9c27c6e5246fdb0cf386358a822cd` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:16.632560 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa42d69-2946-4aac-9ed7-928b74c41167 None admin] RBAC: Authorizing `identity:create_grant(project_id=927057f9a5af4833aeac9992203a223c, user_id=37ed5f72dc8c4a5685bd1245cadcdf7e, role_id=adb9c27c6e5246fdb0cf386358a822cd)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:16.634092 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa42d69-2946-4aac-9ed7-928b74c41167 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:16.701216 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 410/817] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 17 16:41:16 2022] PUT /identity/v3/projects/927057f9a5af4833aeac9992203a223c/users/37ed5f72dc8c4a5685bd1245cadcdf7e/roles/adb9c27c6e5246fdb0cf386358a822cd => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 17 16:41:16.702234 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa42d69-2946-4aac-9ed7-928b74c41167 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:16.721194 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917dbefb-0211-4b07-afcc-f72ccf9ae2ac None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:16.779135 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d9c80-c8b8-4160-a223-b6937d4444de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:16.780984 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d9c80-c8b8-4160-a223-b6937d4444de None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:16.781283 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d9c80-c8b8-4160-a223-b6937d4444de None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:16.781727 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d9c80-c8b8-4160-a223-b6937d4444de None admin] PATH_INFO: `/v3/projects/service` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:16.790697 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268d9c80-c8b8-4160-a223-b6937d4444de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:16.792162 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268d9c80-c8b8-4160-a223-b6937d4444de None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:16.809422 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-268d9c80-c8b8-4160-a223-b6937d4444de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:41:16.809422 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:16.809422 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:16.809422 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:16.809422 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:16.809422 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:16.809422 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:16.809422 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:16.809422 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:16.809422 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:16.809422 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:16.809422 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:16.809422 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:41:16.809422 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:41:16.809422 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:41:16.810406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:41:16.810406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:16.810406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:16.810406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:16.810406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:16.810406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:16.810406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:16.810406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:16.810406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:16.810406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:16.810406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:16.810406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:16.810406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:16.810406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:16.810406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:16.810406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:16.810406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:16.811939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:41:16.811939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:41:16.811939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:41:16.811939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:41:16.811939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:41:16.811939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:41:16.811939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:41:16.811939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:16.811939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 408/818] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:41:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:16.813056 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d9c80-c8b8-4160-a223-b6937d4444de None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:16.828497 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:16.879940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa42d69-2946-4aac-9ed7-928b74c41167 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:16.891039 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:16.927280 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2653b1aa-303d-4056-8ba6-ffab5ddb36c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:16.928404 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653b1aa-303d-4056-8ba6-ffab5ddb36c2 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:16.928910 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653b1aa-303d-4056-8ba6-ffab5ddb36c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:16.929316 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653b1aa-303d-4056-8ba6-ffab5ddb36c2 None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:16.930365 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2653b1aa-303d-4056-8ba6-ffab5ddb36c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:16.930801 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2653b1aa-303d-4056-8ba6-ffab5ddb36c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:16.932765 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2653b1aa-303d-4056-8ba6-ffab5ddb36c2 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:16.951175 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 411/819] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:41:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:16.951592 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2653b1aa-303d-4056-8ba6-ffab5ddb36c2 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:16.963224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d9c80-c8b8-4160-a223-b6937d4444de None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:17.021864 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d48fc-48ed-4872-ac71-9037d9b90ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:17.022555 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d48fc-48ed-4872-ac71-9037d9b90ac6 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:17.022843 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d48fc-48ed-4872-ac71-9037d9b90ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:17.023125 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d48fc-48ed-4872-ac71-9037d9b90ac6 None admin] PATH_INFO: `/v3/services` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:17.023927 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d48fc-48ed-4872-ac71-9037d9b90ac6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:17.024163 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d48fc-48ed-4872-ac71-9037d9b90ac6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:17.025604 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d48fc-48ed-4872-ac71-9037d9b90ac6 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:17.039188 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 409/820] 10.222.0.40 () {56 vars in 1124 bytes} [Sun Jul 17 16:41:16 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:17.039188 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d48fc-48ed-4872-ac71-9037d9b90ac6 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:17.131869 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2653b1aa-303d-4056-8ba6-ffab5ddb36c2 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:17.183377 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:17.190183 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7dd54a-6b9b-4190-97cf-1d1a5423d261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:17.190871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7dd54a-6b9b-4190-97cf-1d1a5423d261 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:17.191160 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7dd54a-6b9b-4190-97cf-1d1a5423d261 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:17.191449 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7dd54a-6b9b-4190-97cf-1d1a5423d261 None admin] PATH_INFO: `/v3/projects` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:17.192251 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7dd54a-6b9b-4190-97cf-1d1a5423d261 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:17.192570 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7dd54a-6b9b-4190-97cf-1d1a5423d261 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:17.194339 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7dd54a-6b9b-4190-97cf-1d1a5423d261 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:17.200387 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d48fc-48ed-4872-ac71-9037d9b90ac6 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:17.212875 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 412/821] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:41:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:17.212875 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7dd54a-6b9b-4190-97cf-1d1a5423d261 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:17.228137 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:17.249130 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f47414-91b7-4c8c-8caf-50bc8d78e7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:17.250256 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f47414-91b7-4c8c-8caf-50bc8d78e7ba None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:17.250554 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f47414-91b7-4c8c-8caf-50bc8d78e7ba None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:17.250838 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f47414-91b7-4c8c-8caf-50bc8d78e7ba None admin] PATH_INFO: `/v3/users/nova` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:17.259853 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f47414-91b7-4c8c-8caf-50bc8d78e7ba None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:17.261953 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f47414-91b7-4c8c-8caf-50bc8d78e7ba None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:17.288168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-e7f47414-91b7-4c8c-8caf-50bc8d78e7ba None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 17 16:41:17.288168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:17.288168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:17.288168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:17.288168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:17.288168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:17.288168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:17.288168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:17.288168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:17.288168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:17.288168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:17.288168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:17.288168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:41:17.288168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:41:17.288168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:41:17.288168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:41:17.289302 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:17.289302 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:17.289302 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:41:17.289302 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:41:17.289302 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:41:17.289302 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:41:17.289302 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:17.289302 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:17.289302 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:17.289302 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:17.289302 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:17.289302 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:17.289302 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:17.289302 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:17.289302 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:17.289302 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:17.289302 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:17.289302 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:17.290262 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:17.290262 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:17.290262 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:41:17.290262 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:41:17.290262 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:41:17.290262 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:41:17.290262 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:41:17.290262 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:41:17.290262 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 17 16:41:17.290262 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:17.290262 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 410/822] 10.222.0.40 () {56 vars in 1093 bytes} [Sun Jul 17 16:41:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:17.291533 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f47414-91b7-4c8c-8caf-50bc8d78e7ba None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:17.386607 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7dd54a-6b9b-4190-97cf-1d1a5423d261 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:17.429418 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90234886-b476-4456-85b6-5ae522fd9780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:17.430112 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90234886-b476-4456-85b6-5ae522fd9780 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:17.430403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90234886-b476-4456-85b6-5ae522fd9780 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:17.430692 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90234886-b476-4456-85b6-5ae522fd9780 None admin] PATH_INFO: `/v3/services` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:17.431420 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90234886-b476-4456-85b6-5ae522fd9780 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:17.431730 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90234886-b476-4456-85b6-5ae522fd9780 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:17.433206 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90234886-b476-4456-85b6-5ae522fd9780 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:17.446615 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 413/823] 10.222.0.40 () {56 vars in 1124 bytes} [Sun Jul 17 16:41:17 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:17.446615 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90234886-b476-4456-85b6-5ae522fd9780 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:17.460841 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f47414-91b7-4c8c-8caf-50bc8d78e7ba None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:17.520489 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca094472-5d08-4a87-b16b-bebbfd944d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:17.521857 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca094472-5d08-4a87-b16b-bebbfd944d32 None admin] REQUEST_METHOD: `PUT` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:17.522155 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca094472-5d08-4a87-b16b-bebbfd944d32 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:17.522448 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca094472-5d08-4a87-b16b-bebbfd944d32 None admin] PATH_INFO: `/v3/projects/927057f9a5af4833aeac9992203a223c/users/69482eeb1e8547aabca9f036b5c34f13/roles/1025051975f64f689effa58dc4259e36` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:17.524633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca094472-5d08-4a87-b16b-bebbfd944d32 None admin] RBAC: Authorizing `identity:create_grant(project_id=927057f9a5af4833aeac9992203a223c, user_id=69482eeb1e8547aabca9f036b5c34f13, role_id=1025051975f64f689effa58dc4259e36)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:17.526153 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca094472-5d08-4a87-b16b-bebbfd944d32 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:17.533614 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:17.614875 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90234886-b476-4456-85b6-5ae522fd9780 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:17.619486 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 411/824] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 17 16:41:17 2022] PUT /identity/v3/projects/927057f9a5af4833aeac9992203a223c/users/69482eeb1e8547aabca9f036b5c34f13/roles/1025051975f64f689effa58dc4259e36 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 17 16:41:17.620709 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca094472-5d08-4a87-b16b-bebbfd944d32 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:17.697414 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:17.706891 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6519e066-4dc5-47ba-be3c-d69228b8c0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:17.708802 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519e066-4dc5-47ba-be3c-d69228b8c0a4 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:17.709564 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519e066-4dc5-47ba-be3c-d69228b8c0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:17.710534 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519e066-4dc5-47ba-be3c-d69228b8c0a4 None admin] PATH_INFO: `/v3/users` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:17.713198 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6519e066-4dc5-47ba-be3c-d69228b8c0a4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:17.714068 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6519e066-4dc5-47ba-be3c-d69228b8c0a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:17.717742 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6519e066-4dc5-47ba-be3c-d69228b8c0a4 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:17.764825 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 414/825] 10.222.0.40 () {56 vars in 1133 bytes} [Sun Jul 17 16:41:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:17.770164 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6519e066-4dc5-47ba-be3c-d69228b8c0a4 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:17.823509 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca094472-5d08-4a87-b16b-bebbfd944d32 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:17.894070 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a9fc9e-b9d6-48f1-a595-4f3723214454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:17.894827 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9fc9e-b9d6-48f1-a595-4f3723214454 None admin] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:17.895205 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9fc9e-b9d6-48f1-a595-4f3723214454 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:17.895570 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a9fc9e-b9d6-48f1-a595-4f3723214454 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:17.896502 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a9fc9e-b9d6-48f1-a595-4f3723214454 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:17.898972 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a9fc9e-b9d6-48f1-a595-4f3723214454 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:17.994777 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6519e066-4dc5-47ba-be3c-d69228b8c0a4 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:17.999458 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 412/826] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 17 16:41:17 2022] POST /identity/v3/endpoints => generated 346 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:41:18.028657 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:18.083819 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59d3be-51b8-4f35-9cd3-836bd796b7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:18.085026 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59d3be-51b8-4f35-9cd3-836bd796b7e7 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:18.085538 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59d3be-51b8-4f35-9cd3-836bd796b7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:18.086041 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59d3be-51b8-4f35-9cd3-836bd796b7e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:18.213387 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd59d3be-51b8-4f35-9cd3-836bd796b7e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:18.215139 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd59d3be-51b8-4f35-9cd3-836bd796b7e7 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:18.336905 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 415/827] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:41:17 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 17 16:41:18.346565 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a9fc9e-b9d6-48f1-a595-4f3723214454 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:18.543050 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:18.563691 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a9fc9e-b9d6-48f1-a595-4f3723214454 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:18.618356 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7974a9-627c-4d5c-a82e-afac2f074bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:18.619569 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7974a9-627c-4d5c-a82e-afac2f074bb4 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:18.620071 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7974a9-627c-4d5c-a82e-afac2f074bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:18.620594 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7974a9-627c-4d5c-a82e-afac2f074bb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:18.633694 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7974a9-627c-4d5c-a82e-afac2f074bb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:18.635614 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7974a9-627c-4d5c-a82e-afac2f074bb4 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:18.658131 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-5e7974a9-627c-4d5c-a82e-afac2f074bb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:18.658131 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:18.658131 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:18.658131 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:18.658131 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:18.658131 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:18.658131 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:18.658131 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:18.658131 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:18.658131 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:18.658131 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:18.658131 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:18.658131 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:41:18.658131 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:41:18.658131 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:41:18.659122 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:41:18.659122 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:18.659122 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:18.659122 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:18.659122 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:18.659122 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:18.659122 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:18.659122 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:18.659122 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:18.659122 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:18.659122 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:18.659122 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:18.659122 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:18.659122 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:18.659122 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:18.659122 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:18.659122 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:18.659994 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:41:18.659994 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:41:18.659994 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:18.659994 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:18.662452 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 413/828] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:18.674315 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59d3be-51b8-4f35-9cd3-836bd796b7e7 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:18.692676 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:18.863588 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59d3be-51b8-4f35-9cd3-836bd796b7e7 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:18.929008 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fc0de-3f8c-48f2-9690-fdb0a9ad4083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:18.930152 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fc0de-3f8c-48f2-9690-fdb0a9ad4083 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:18.930643 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fc0de-3f8c-48f2-9690-fdb0a9ad4083 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:18.931153 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fc0de-3f8c-48f2-9690-fdb0a9ad4083 None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:18.932508 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fc0de-3f8c-48f2-9690-fdb0a9ad4083 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:18.933080 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fc0de-3f8c-48f2-9690-fdb0a9ad4083 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:18.935722 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fc0de-3f8c-48f2-9690-fdb0a9ad4083 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:18.961654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 416/829] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:41:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:18.975150 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7974a9-627c-4d5c-a82e-afac2f074bb4 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:19.168038 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7974a9-627c-4d5c-a82e-afac2f074bb4 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:19.241229 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfda8a3-7300-425d-81ff-e8109b950ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:19.242763 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfda8a3-7300-425d-81ff-e8109b950ff8 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:19.243258 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfda8a3-7300-425d-81ff-e8109b950ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:19.243757 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfda8a3-7300-425d-81ff-e8109b950ff8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:19.258579 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfda8a3-7300-425d-81ff-e8109b950ff8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:19.261397 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfda8a3-7300-425d-81ff-e8109b950ff8 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:19.289811 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-7bfda8a3-7300-425d-81ff-e8109b950ff8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:41:19.289811 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:19.289811 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:19.289811 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:19.289811 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:19.289811 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:19.289811 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:19.289811 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:19.289811 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:19.289811 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:19.289811 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:19.289811 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:19.289811 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:41:19.289811 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:41:19.289811 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:41:19.290793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:41:19.290793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:19.290793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:19.290793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:19.290793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:19.290793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:19.290793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:19.290793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:19.290793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:19.290793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:19.290793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:19.290793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:19.290793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:19.290793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:19.290793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:19.290793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:19.290793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:19.291707 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:41:19.291707 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:41:19.291707 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:41:19.291707 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:41:19.291707 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:41:19.291707 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:41:19.291707 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:41:19.291707 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:19.292641 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 414/830] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:41:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:19.299469 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fc0de-3f8c-48f2-9690-fdb0a9ad4083 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:19.418366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e6f45-ff50-4215-ab20-f8fe8a1b6df1 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:19.418874 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e6f45-ff50-4215-ab20-f8fe8a1b6df1 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:19.419289 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e6f45-ff50-4215-ab20-f8fe8a1b6df1 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:19.422571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 415/831] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:19.428429 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684734b-1478-4587-a367-757f54d7cb14 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:19.428766 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684734b-1478-4587-a367-757f54d7cb14 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:19.429050 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684734b-1478-4587-a367-757f54d7cb14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:19.436431 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:19.448149 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:19.450462 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-f684734b-1478-4587-a367-757f54d7cb14 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:19.459280 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-f684734b-1478-4587-a367-757f54d7cb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:19.507541 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fc0de-3f8c-48f2-9690-fdb0a9ad4083 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:19.548952 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 416/832] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 17 16:41:19.574663 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684734b-1478-4587-a367-757f54d7cb14 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:19.585994 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d0a1d2-a78a-46ae-89a3-dbf152d25c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:19.587043 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0a1d2-a78a-46ae-89a3-dbf152d25c71 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:19.587539 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0a1d2-a78a-46ae-89a3-dbf152d25c71 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:19.588024 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0a1d2-a78a-46ae-89a3-dbf152d25c71 None admin] PATH_INFO: `/v3/projects` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:19.589124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d0a1d2-a78a-46ae-89a3-dbf152d25c71 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:19.589758 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d0a1d2-a78a-46ae-89a3-dbf152d25c71 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:19.591458 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d0a1d2-a78a-46ae-89a3-dbf152d25c71 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:19.611407 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 417/833] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:41:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:19.615184 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d0a1d2-a78a-46ae-89a3-dbf152d25c71 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:19.738731 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d0a1d2-a78a-46ae-89a3-dbf152d25c71 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:19.766056 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f684734b-1478-4587-a367-757f54d7cb14 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:19.775378 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:19.783349 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ff481-9b38-4e32-97ec-0a6c5750e6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:19.784861 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ff481-9b38-4e32-97ec-0a6c5750e6ad None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:19.791233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ff481-9b38-4e32-97ec-0a6c5750e6ad None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:19.791233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ff481-9b38-4e32-97ec-0a6c5750e6ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:19.791233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ff481-9b38-4e32-97ec-0a6c5750e6ad None admin] RBAC: Adding query filter params (role.id=1025051975f64f689effa58dc4259e36, scope.project.id=927057f9a5af4833aeac9992203a223c, user.id=03244087f1a34bddb11ea46aba11a148) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:19.791233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ff481-9b38-4e32-97ec-0a6c5750e6ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:19.791233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ff481-9b38-4e32-97ec-0a6c5750e6ad None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:19.801017 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 418/834] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:41:19 2022] GET /identity/v3/role_assignments?user.id=03244087f1a34bddb11ea46aba11a148&scope.project.id=927057f9a5af4833aeac9992203a223c&role.id=1025051975f64f689effa58dc4259e36 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:41:19.836430 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b6b2d-e74b-4275-88d5-0f0bd29973dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:19.837876 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b6b2d-e74b-4275-88d5-0f0bd29973dc None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:19.838353 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b6b2d-e74b-4275-88d5-0f0bd29973dc None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:19.838837 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b6b2d-e74b-4275-88d5-0f0bd29973dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:19.843149 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:19.862767 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1b6b2d-e74b-4275-88d5-0f0bd29973dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:41:19.878725 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-4c1b6b2d-e74b-4275-88d5-0f0bd29973dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:41:19.878725 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:19.878725 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:19.878725 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:19.878725 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:19.878725 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:19.878725 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:19.878725 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:19.878725 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:19.878725 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:19.878725 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:19.878725 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:19.878725 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:41:19.878725 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:41:19.878725 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:41:19.878725 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:41:19.879807 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:41:19.879807 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:41:19.879807 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:41:19.879807 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:41:19.879807 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:19.879807 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:19.879807 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:19.879807 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:19.879807 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:19.879807 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:19.879807 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:19.879807 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:19.879807 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:19.879807 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:19.879807 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:19.879807 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:19.881110 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:19.881110 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:19.881110 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:19.881110 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:19.881110 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:41:19.881110 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:41:19.881110 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:41:19.881110 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:41:19.881110 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:41:19.881110 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:41:19.881110 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:41:19.881110 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:19.882171 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 417/835] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 17 16:41:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:41:19.890151 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ff481-9b38-4e32-97ec-0a6c5750e6ad None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:20.020291 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ff481-9b38-4e32-97ec-0a6c5750e6ad None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:20.060078 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0462a77-6cca-419a-a66c-6221c95ee644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:20.061564 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0462a77-6cca-419a-a66c-6221c95ee644 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:20.062047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0462a77-6cca-419a-a66c-6221c95ee644 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:20.062545 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0462a77-6cca-419a-a66c-6221c95ee644 None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:20.063992 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0462a77-6cca-419a-a66c-6221c95ee644 None admin] RBAC: Adding query filter params (name=service) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:20.064560 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0462a77-6cca-419a-a66c-6221c95ee644 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:20.067212 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0462a77-6cca-419a-a66c-6221c95ee644 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:20.085578 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 419/836] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 17 16:41:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:41:20.091646 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b6b2d-e74b-4275-88d5-0f0bd29973dc None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:20.299651 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b6b2d-e74b-4275-88d5-0f0bd29973dc None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:20.341838 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6f34b-c454-40a9-8d3d-7c645662d317 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:20.342879 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6f34b-c454-40a9-8d3d-7c645662d317 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:20.343398 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6f34b-c454-40a9-8d3d-7c645662d317 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:20.353057 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 420/837] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:41:20.355543 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23bcb5-bdae-451e-ac95-4ac4526d0069 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:20.356064 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23bcb5-bdae-451e-ac95-4ac4526d0069 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:20.356578 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23bcb5-bdae-451e-ac95-4ac4526d0069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:20.370244 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa716fc-b630-4eb8-8d92-36222b648cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:20.371687 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa716fc-b630-4eb8-8d92-36222b648cb4 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:20.372194 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa716fc-b630-4eb8-8d92-36222b648cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:20.372698 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa716fc-b630-4eb8-8d92-36222b648cb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:20.386505 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-8c23bcb5-bdae-451e-ac95-4ac4526d0069 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:20.389683 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-8c23bcb5-bdae-451e-ac95-4ac4526d0069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:20.396875 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa716fc-b630-4eb8-8d92-36222b648cb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:20.399836 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa716fc-b630-4eb8-8d92-36222b648cb4 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:20.437159 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-2fa716fc-b630-4eb8-8d92-36222b648cb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:20.437159 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:20.437159 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:20.437159 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:20.437159 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:20.437159 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:20.437159 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:20.437159 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:20.437159 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:20.437159 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:20.437159 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:20.437159 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:20.437159 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:41:20.437159 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:41:20.437159 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:41:20.438647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:41:20.438647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:20.438647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:20.438647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:20.438647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:20.438647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:20.438647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:20.438647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:20.438647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:20.438647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:20.438647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:20.438647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:20.438647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:20.438647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:20.438647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:20.438647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:20.438647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:20.439543 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:41:20.439543 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:41:20.439543 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:20.439543 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:20.440066 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 418/838] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:20.449822 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa716fc-b630-4eb8-8d92-36222b648cb4 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:20.499123 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:20.531768 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:20.535662 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 421/839] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:20 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 17 16:41:20.579421 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c23bcb5-bdae-451e-ac95-4ac4526d0069 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:20.625343 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa716fc-b630-4eb8-8d92-36222b648cb4 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:20.671379 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4933594-c302-495c-a924-6599afc50ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:20.672752 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4933594-c302-495c-a924-6599afc50ea1 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:20.673281 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4933594-c302-495c-a924-6599afc50ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:20.673789 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4933594-c302-495c-a924-6599afc50ea1 None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:20.675047 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4933594-c302-495c-a924-6599afc50ea1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:20.675570 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4933594-c302-495c-a924-6599afc50ea1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:20.678275 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4933594-c302-495c-a924-6599afc50ea1 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:20.696103 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 419/840] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:41:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:41:20.704984 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4933594-c302-495c-a924-6599afc50ea1 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:20.752344 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c23bcb5-bdae-451e-ac95-4ac4526d0069 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:20.803024 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9317ec-1b72-4448-a59b-816266e13245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:20.804275 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9317ec-1b72-4448-a59b-816266e13245 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:20.804818 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9317ec-1b72-4448-a59b-816266e13245 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:20.806100 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9317ec-1b72-4448-a59b-816266e13245 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:20.823061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9317ec-1b72-4448-a59b-816266e13245 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:41:20.833494 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-0a9317ec-1b72-4448-a59b-816266e13245 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:41:20.833494 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:20.833494 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:20.833494 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:20.833494 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:20.833494 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:20.833494 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:20.833494 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:20.833494 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:20.833494 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:20.833494 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:20.833494 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:20.833494 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:41:20.833494 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:41:20.833494 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:41:20.833494 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:41:20.834460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:41:20.834460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:41:20.834460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:41:20.834460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:41:20.834460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:20.834460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:20.834460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:20.834460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:20.834460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:20.834460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:20.834460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:20.834460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:20.834460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:20.834460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:20.834460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:20.834460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:20.835387 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:20.835387 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:20.835387 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:20.835387 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:20.835387 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:41:20.835387 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:41:20.835387 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:41:20.835387 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:41:20.835387 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:41:20.835387 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:41:20.835387 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:41:20.835387 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:20.843283 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 422/841] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:41:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:20.846724 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9317ec-1b72-4448-a59b-816266e13245 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:20.907704 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4933594-c302-495c-a924-6599afc50ea1 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:20.919581 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:20.928297 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:20.963269 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cac0c9-1829-42dc-9132-677d06616ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:20.964424 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cac0c9-1829-42dc-9132-677d06616ed2 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:20.964926 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cac0c9-1829-42dc-9132-677d06616ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:20.965430 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cac0c9-1829-42dc-9132-677d06616ed2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:20.977634 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cac0c9-1829-42dc-9132-677d06616ed2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:20.979796 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cac0c9-1829-42dc-9132-677d06616ed2 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:21.003068 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-f5cac0c9-1829-42dc-9132-677d06616ed2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 17 16:41:21.003068 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:21.003068 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:21.003068 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:21.003068 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:21.003068 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:21.003068 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:21.003068 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:21.003068 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:21.003068 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:21.003068 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:21.003068 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:21.003068 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:41:21.003068 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:41:21.003068 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:41:21.003068 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:41:21.005324 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:21.005324 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:21.005324 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:41:21.005324 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:41:21.005324 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:41:21.005324 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:41:21.005324 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:21.005324 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:21.005324 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:21.005324 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:21.005324 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:21.005324 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:21.005324 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:21.005324 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:21.005324 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:21.005324 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:21.005324 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:21.005324 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:21.007178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:21.007178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:21.007178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:41:21.007178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:41:21.007178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:41:21.007178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:41:21.007178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:41:21.007178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:41:21.007178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 17 16:41:21.007178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:21.007178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 420/842] 10.222.0.40 () {56 vars in 1117 bytes} [Sun Jul 17 16:41:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:21.007178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c912c-97c2-4d8d-9721-f0728dc09ae5 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:21.007178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c912c-97c2-4d8d-9721-f0728dc09ae5 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:21.008514 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c912c-97c2-4d8d-9721-f0728dc09ae5 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:21.012834 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 421/843] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:21.014611 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468c912c-97c2-4d8d-9721-f0728dc09ae5 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:21.037737 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9317ec-1b72-4448-a59b-816266e13245 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:21.106043 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc0f23-daaa-4e3b-a698-c8b373d8ba96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:21.107098 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc0f23-daaa-4e3b-a698-c8b373d8ba96 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:21.108004 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc0f23-daaa-4e3b-a698-c8b373d8ba96 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:21.108433 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc0f23-daaa-4e3b-a698-c8b373d8ba96 None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:21.109755 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fc0f23-daaa-4e3b-a698-c8b373d8ba96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:21.110323 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fc0f23-daaa-4e3b-a698-c8b373d8ba96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:21.113118 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fc0f23-daaa-4e3b-a698-c8b373d8ba96 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:21.130562 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 423/844] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:41:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:21.132771 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32e8e7-804e-442c-b112-b2bedbcd9932 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:21.133312 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32e8e7-804e-442c-b112-b2bedbcd9932 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:21.133798 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32e8e7-804e-442c-b112-b2bedbcd9932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:21.162363 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-5f32e8e7-804e-442c-b112-b2bedbcd9932 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:21.164281 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-5f32e8e7-804e-442c-b112-b2bedbcd9932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:21.170035 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468c912c-97c2-4d8d-9721-f0728dc09ae5 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:21.235315 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d4388-0d94-4b16-9878-23067a5566d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:21.237517 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d4388-0d94-4b16-9878-23067a5566d8 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:21.237980 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d4388-0d94-4b16-9878-23067a5566d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:21.238434 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d4388-0d94-4b16-9878-23067a5566d8 None admin] PATH_INFO: `/v3/users` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:21.239917 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d4388-0d94-4b16-9878-23067a5566d8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:21.240431 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d4388-0d94-4b16-9878-23067a5566d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:21.243006 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d4388-0d94-4b16-9878-23067a5566d8 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:21.284819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 422/845] 10.222.0.40 () {56 vars in 1149 bytes} [Sun Jul 17 16:41:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:21.284819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d4388-0d94-4b16-9878-23067a5566d8 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:21.301111 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 424/846] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:21 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:41:21.302693 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32e8e7-804e-442c-b112-b2bedbcd9932 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:21.342975 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:21.385362 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:21.451877 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d4388-0d94-4b16-9878-23067a5566d8 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:21.495351 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2875dff-f07d-4793-873a-6a3ece02ab2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:21.496897 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2875dff-f07d-4793-873a-6a3ece02ab2f None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:21.497417 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2875dff-f07d-4793-873a-6a3ece02ab2f None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:21.497920 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2875dff-f07d-4793-873a-6a3ece02ab2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:21.498078 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32e8e7-804e-442c-b112-b2bedbcd9932 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:21.520148 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2875dff-f07d-4793-873a-6a3ece02ab2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:21.521773 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2875dff-f07d-4793-873a-6a3ece02ab2f None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:21.542870 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-f2875dff-f07d-4793-873a-6a3ece02ab2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:21.542870 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:21.542870 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:21.542870 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:21.542870 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:21.542870 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:21.542870 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:21.542870 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:21.542870 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:21.542870 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:21.542870 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:21.542870 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:21.542870 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:41:21.542870 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:41:21.542870 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:41:21.544810 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:41:21.544810 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:21.544810 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:21.544810 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:21.544810 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:21.544810 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:21.544810 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:21.544810 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:21.544810 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:21.544810 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:21.544810 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:21.544810 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:21.544810 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:21.544810 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:21.544810 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:21.544810 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:21.544810 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:21.546812 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:41:21.546812 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:41:21.546812 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:21.546812 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:21.549214 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 423/847] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:21.551481 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2875dff-f07d-4793-873a-6a3ece02ab2f None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:21.562476 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3def6c-a3e8-4e05-a5a0-c45993224e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:21.563613 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3def6c-a3e8-4e05-a5a0-c45993224e1a None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:21.564102 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3def6c-a3e8-4e05-a5a0-c45993224e1a None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:21.564595 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3def6c-a3e8-4e05-a5a0-c45993224e1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:21.668315 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:21.688375 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3def6c-a3e8-4e05-a5a0-c45993224e1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:21.691086 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3def6c-a3e8-4e05-a5a0-c45993224e1a None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:21.697470 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2875dff-f07d-4793-873a-6a3ece02ab2f None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:21.744650 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c29a0-c70c-43ad-a56d-8ac03b497ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:21.745537 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c29a0-c70c-43ad-a56d-8ac03b497ce1 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:21.745818 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c29a0-c70c-43ad-a56d-8ac03b497ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:21.746094 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c29a0-c70c-43ad-a56d-8ac03b497ce1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:21.746927 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1c29a0-c70c-43ad-a56d-8ac03b497ce1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:21.749424 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1c29a0-c70c-43ad-a56d-8ac03b497ce1 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:21.763330 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:21.770530 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-6e1c29a0-c70c-43ad-a56d-8ac03b497ce1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 17 16:41:21.770530 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:21.770530 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 17 16:41:21.770530 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 17 16:41:21.770530 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 17 16:41:21.770530 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 17 16:41:21.770530 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 17 16:41:21.770530 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:21.770530 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 17 16:41:21.770530 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:21.770530 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 17 16:41:21.770530 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:21.770530 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:21.770530 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:21.770530 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:21.770530 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:21.770530 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:21.771716 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:21.771716 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:21.771716 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:21.771716 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:21.771716 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:21.771716 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:21.771716 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 17 16:41:21.771716 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 17 16:41:21.771716 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 17 16:41:21.771716 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 17 16:41:21.771716 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:21.771716 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:21.771716 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:21.771716 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:21.771716 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:21.771716 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:21.772582 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:21.772582 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:21.772582 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:21.772582 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:21.772582 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:21.772582 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:21.772582 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:21.772582 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:21.772582 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:21.772582 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:21.772582 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 17 16:41:21.772582 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:21.772582 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 17 16:41:21.772582 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:21.774124 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 424/848] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 17 16:41:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:21.774980 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c29a0-c70c-43ad-a56d-8ac03b497ce1 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:21.873896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 425/849] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:41:21 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 17 16:41:21.875373 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3def6c-a3e8-4e05-a5a0-c45993224e1a None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:21.957814 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c29a0-c70c-43ad-a56d-8ac03b497ce1 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:22.025883 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3def6c-a3e8-4e05-a5a0-c45993224e1a None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:22.026374 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de18a66e-4e25-4606-8d71-60c62f7353bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:22.027464 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18a66e-4e25-4606-8d71-60c62f7353bd None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:22.027962 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18a66e-4e25-4606-8d71-60c62f7353bd None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:22.028462 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18a66e-4e25-4606-8d71-60c62f7353bd None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:22.031105 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de18a66e-4e25-4606-8d71-60c62f7353bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:22.031683 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de18a66e-4e25-4606-8d71-60c62f7353bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:22.034384 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de18a66e-4e25-4606-8d71-60c62f7353bd None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:22.052091 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:22.058392 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 425/850] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:41:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:22.060016 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de18a66e-4e25-4606-8d71-60c62f7353bd None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:22.078422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d977fe1-1db9-437c-bd02-3b5bec96c5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:22.079834 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d977fe1-1db9-437c-bd02-3b5bec96c5ba None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:22.080288 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d977fe1-1db9-437c-bd02-3b5bec96c5ba None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:22.080719 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d977fe1-1db9-437c-bd02-3b5bec96c5ba None admin] PATH_INFO: `/v3/services` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:22.081810 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d977fe1-1db9-437c-bd02-3b5bec96c5ba None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:22.082349 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d977fe1-1db9-437c-bd02-3b5bec96c5ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:22.083925 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d977fe1-1db9-437c-bd02-3b5bec96c5ba None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:22.090144 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 426/851] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 17 16:41:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:22.091423 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d977fe1-1db9-437c-bd02-3b5bec96c5ba None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:22.099618 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:22.212439 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de18a66e-4e25-4606-8d71-60c62f7353bd None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:22.230488 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d977fe1-1db9-437c-bd02-3b5bec96c5ba None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:22.255836 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ca5a7-63bd-4c15-b059-88cd31f9eb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:22.257181 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ca5a7-63bd-4c15-b059-88cd31f9eb9a None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:22.257691 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ca5a7-63bd-4c15-b059-88cd31f9eb9a None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:22.258167 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786ca5a7-63bd-4c15-b059-88cd31f9eb9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:22.273433 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786ca5a7-63bd-4c15-b059-88cd31f9eb9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:22.275882 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786ca5a7-63bd-4c15-b059-88cd31f9eb9a None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:22.279610 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d78fd3-e902-41d1-8bad-2817cc56d028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:22.280447 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d78fd3-e902-41d1-8bad-2817cc56d028 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:22.280853 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d78fd3-e902-41d1-8bad-2817cc56d028 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:22.281334 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d78fd3-e902-41d1-8bad-2817cc56d028 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:22.290942 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d78fd3-e902-41d1-8bad-2817cc56d028 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:22.292457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d78fd3-e902-41d1-8bad-2817cc56d028 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:22.308457 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-786ca5a7-63bd-4c15-b059-88cd31f9eb9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:22.308457 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:22.308457 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:22.308457 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:22.308457 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:22.308457 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:22.308457 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:22.308457 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:22.308457 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:22.308457 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:22.308457 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:22.308457 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:22.308457 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:41:22.308457 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:41:22.308457 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:41:22.309711 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:41:22.309711 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:22.309711 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:22.309711 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:22.309711 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:22.309711 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:22.309711 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:22.309711 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:22.309711 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:22.309711 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:22.309711 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:22.309711 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:22.309711 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:22.309711 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:22.309711 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:22.309711 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:22.309711 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:22.310942 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:41:22.310942 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:41:22.310942 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:22.310942 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:22.311166 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-a5d78fd3-e902-41d1-8bad-2817cc56d028 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 17 16:41:22.311166 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:22.311166 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:22.311166 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:22.311166 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:22.311166 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:22.311166 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:22.311166 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:22.311166 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:22.311166 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:22.311166 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:22.311166 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:22.311166 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:41:22.311166 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:41:22.311166 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:41:22.311166 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:41:22.313143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:22.313143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:22.313143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:41:22.313143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:41:22.313143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:41:22.313143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:41:22.313143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:22.313143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:22.313143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:22.313143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:22.313143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:22.313143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:22.313143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:22.313143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:22.313143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:22.313143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:22.313143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:22.313143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:22.318139 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:22.318139 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:22.318139 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:41:22.318139 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:41:22.318139 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:41:22.318139 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:41:22.318139 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:41:22.318139 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:41:22.318139 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 17 16:41:22.318139 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:22.319482 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 426/852] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:22.319482 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ca5a7-63bd-4c15-b059-88cd31f9eb9a None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:22.319785 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 427/853] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:41:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:22.319785 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d78fd3-e902-41d1-8bad-2817cc56d028 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:22.385443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:22.405762 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:22.449163 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786ca5a7-63bd-4c15-b059-88cd31f9eb9a None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:22.451836 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d78fd3-e902-41d1-8bad-2817cc56d028 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:22.490977 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd0544-6ea7-4423-88d0-78985cfb17ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:22.491648 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd0544-6ea7-4423-88d0-78985cfb17ef None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:22.491818 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd0544-6ea7-4423-88d0-78985cfb17ef None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:22.492082 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd0544-6ea7-4423-88d0-78985cfb17ef None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:22.492866 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd0544-6ea7-4423-88d0-78985cfb17ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:22.493165 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd0544-6ea7-4423-88d0-78985cfb17ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:22.494547 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd0544-6ea7-4423-88d0-78985cfb17ef None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:22.499396 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaadf914-4ca5-4b4a-ade0-a52f130c68b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:22.500247 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaadf914-4ca5-4b4a-ade0-a52f130c68b0 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:22.500531 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaadf914-4ca5-4b4a-ade0-a52f130c68b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:22.500850 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaadf914-4ca5-4b4a-ade0-a52f130c68b0 None admin] PATH_INFO: `/v3/services` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:22.501711 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaadf914-4ca5-4b4a-ade0-a52f130c68b0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:22.502031 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaadf914-4ca5-4b4a-ade0-a52f130c68b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:22.503453 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaadf914-4ca5-4b4a-ade0-a52f130c68b0 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:22.508584 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 427/854] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 17 16:41:22 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:22.509482 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaadf914-4ca5-4b4a-ade0-a52f130c68b0 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:22.517087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 428/855] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:22.518498 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec970773-3ecd-49fd-874d-6f5244c3eedd None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:22.518790 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec970773-3ecd-49fd-874d-6f5244c3eedd None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:22.519065 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec970773-3ecd-49fd-874d-6f5244c3eedd None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:22.524522 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 429/856] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:22.524522 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec970773-3ecd-49fd-874d-6f5244c3eedd None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:22.655658 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaadf914-4ca5-4b4a-ade0-a52f130c68b0 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:22.658219 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec970773-3ecd-49fd-874d-6f5244c3eedd None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:22.676503 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:22.700920 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:22.703792 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56abca66-3869-4694-9b87-67ec3c894622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:22.705111 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56abca66-3869-4694-9b87-67ec3c894622 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:22.709590 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56abca66-3869-4694-9b87-67ec3c894622 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:22.709590 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56abca66-3869-4694-9b87-67ec3c894622 None admin] PATH_INFO: `/v3/users` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:22.709590 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56abca66-3869-4694-9b87-67ec3c894622 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:22.709590 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56abca66-3869-4694-9b87-67ec3c894622 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:22.710623 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56abca66-3869-4694-9b87-67ec3c894622 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:22.716408 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de422253-08ea-4664-a5ae-aa85bb279c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:22.717616 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de422253-08ea-4664-a5ae-aa85bb279c53 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:22.718118 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de422253-08ea-4664-a5ae-aa85bb279c53 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:22.718632 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de422253-08ea-4664-a5ae-aa85bb279c53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:22.727204 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de422253-08ea-4664-a5ae-aa85bb279c53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:22.728836 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de422253-08ea-4664-a5ae-aa85bb279c53 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:22.747736 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 428/857] 10.222.0.40 () {56 vars in 1135 bytes} [Sun Jul 17 16:41:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:22.747909 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-de422253-08ea-4664-a5ae-aa85bb279c53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:41:22.747909 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:22.747909 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:22.747909 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:22.747909 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:22.747909 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:22.747909 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:22.747909 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:22.747909 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:22.747909 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:22.747909 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:22.747909 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:22.747909 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:41:22.747909 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:41:22.747909 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:41:22.749654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:41:22.749654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:22.749654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:22.749654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:22.749654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:22.749654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:22.749654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:22.749654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:22.749654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:22.749654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:22.749654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:22.749654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:22.749654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:22.749654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:22.749654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:22.749654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:22.749654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:22.750629 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:41:22.750629 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:41:22.750629 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:41:22.750629 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:41:22.750629 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:41:22.750629 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:41:22.750629 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:41:22.750629 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:22.751056 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48abe0-9748-4fa6-b4ce-57a074584029 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:22.751056 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48abe0-9748-4fa6-b4ce-57a074584029 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:22.751192 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 430/858] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:41:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:22.751192 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de422253-08ea-4664-a5ae-aa85bb279c53 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:22.751319 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a48abe0-9748-4fa6-b4ce-57a074584029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:22.775059 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-6a48abe0-9748-4fa6-b4ce-57a074584029 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:22.776293 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-6a48abe0-9748-4fa6-b4ce-57a074584029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:22.889612 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de422253-08ea-4664-a5ae-aa85bb279c53 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:22.893725 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 429/859] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:22 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:41:22.894622 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a48abe0-9748-4fa6-b4ce-57a074584029 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:22.933009 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46ec00-01d6-42cf-be83-3b0f63fd36ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:22.933719 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46ec00-01d6-42cf-be83-3b0f63fd36ea None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:22.934003 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46ec00-01d6-42cf-be83-3b0f63fd36ea None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:22.934296 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46ec00-01d6-42cf-be83-3b0f63fd36ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:23.011310 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:23.033910 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb46ec00-01d6-42cf-be83-3b0f63fd36ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:23.036001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb46ec00-01d6-42cf-be83-3b0f63fd36ea None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:23.051967 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a48abe0-9748-4fa6-b4ce-57a074584029 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:23.055626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:23.096657 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c5964-6d9c-4d5a-84cc-1062308a12ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:23.097474 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c5964-6d9c-4d5a-84cc-1062308a12ac None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:23.097844 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c5964-6d9c-4d5a-84cc-1062308a12ac None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:23.098133 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c5964-6d9c-4d5a-84cc-1062308a12ac None admin] PATH_INFO: `/v3/projects` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:23.098966 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932c5964-6d9c-4d5a-84cc-1062308a12ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:23.099261 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932c5964-6d9c-4d5a-84cc-1062308a12ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:23.100725 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932c5964-6d9c-4d5a-84cc-1062308a12ac None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:23.114026 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 430/860] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:41:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:23.114945 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c5964-6d9c-4d5a-84cc-1062308a12ac None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:23.203022 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 431/861] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:41:22 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 17 16:41:23.203022 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46ec00-01d6-42cf-be83-3b0f63fd36ea None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:23.257616 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c5964-6d9c-4d5a-84cc-1062308a12ac None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:23.305565 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52cb890-a68e-4576-baa4-c76000666b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:23.307066 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52cb890-a68e-4576-baa4-c76000666b42 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:23.307484 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52cb890-a68e-4576-baa4-c76000666b42 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:23.307899 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52cb890-a68e-4576-baa4-c76000666b42 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:23.308133 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:23.309306 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52cb890-a68e-4576-baa4-c76000666b42 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:23.312584 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52cb890-a68e-4576-baa4-c76000666b42 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:23.325970 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-c52cb890-a68e-4576-baa4-c76000666b42 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 17 16:41:23.325970 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:23.325970 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 17 16:41:23.325970 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 17 16:41:23.325970 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 17 16:41:23.325970 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 17 16:41:23.325970 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 17 16:41:23.325970 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:23.325970 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 17 16:41:23.325970 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:23.325970 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 17 16:41:23.325970 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:23.325970 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:23.325970 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:23.325970 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:23.325970 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:23.335667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:23.335667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:23.335667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:23.335667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:23.335667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:23.335667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:23.335667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:23.335667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 17 16:41:23.335667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 17 16:41:23.335667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 17 16:41:23.335667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 17 16:41:23.335667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:23.335667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:23.335667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:23.335667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:23.335667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:23.337640 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:23.337640 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:23.337640 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:23.337640 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:23.337640 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:23.337640 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:23.337640 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:23.337640 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:23.337640 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:23.337640 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:23.337640 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:23.337640 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 17 16:41:23.337640 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:23.337640 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 17 16:41:23.337640 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:23.337640 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 431/862] 10.222.0.40 () {56 vars in 1132 bytes} [Sun Jul 17 16:41:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:23.339499 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52cb890-a68e-4576-baa4-c76000666b42 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:23.350012 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46ec00-01d6-42cf-be83-3b0f63fd36ea None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:23.371199 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:23.402398 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167fdd0-e6cd-4587-a642-549812c3e9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:23.403328 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167fdd0-e6cd-4587-a642-549812c3e9f2 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:23.403615 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167fdd0-e6cd-4587-a642-549812c3e9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:23.403898 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167fdd0-e6cd-4587-a642-549812c3e9f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:23.404739 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6167fdd0-e6cd-4587-a642-549812c3e9f2 None admin] RBAC: Adding query filter params (role.id=adb9c27c6e5246fdb0cf386358a822cd, scope.project.id=927057f9a5af4833aeac9992203a223c, user.id=37ed5f72dc8c4a5685bd1245cadcdf7e) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:23.405086 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6167fdd0-e6cd-4587-a642-549812c3e9f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:23.406643 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6167fdd0-e6cd-4587-a642-549812c3e9f2 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:23.413056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 432/863] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:41:23 2022] GET /identity/v3/role_assignments?user.id=37ed5f72dc8c4a5685bd1245cadcdf7e&scope.project.id=927057f9a5af4833aeac9992203a223c&role.id=adb9c27c6e5246fdb0cf386358a822cd => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:23.423671 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167fdd0-e6cd-4587-a642-549812c3e9f2 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:23.502787 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52cb890-a68e-4576-baa4-c76000666b42 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:23.560316 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33776966-f21a-46d6-ab3a-dd099690ea14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:23.562070 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33776966-f21a-46d6-ab3a-dd099690ea14 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:23.562526 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33776966-f21a-46d6-ab3a-dd099690ea14 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:23.563627 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33776966-f21a-46d6-ab3a-dd099690ea14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:23.581115 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33776966-f21a-46d6-ab3a-dd099690ea14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:23.583000 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33776966-f21a-46d6-ab3a-dd099690ea14 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:23.595935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167fdd0-e6cd-4587-a642-549812c3e9f2 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:23.609441 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-33776966-f21a-46d6-ab3a-dd099690ea14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:23.609441 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:23.609441 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:23.609441 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:23.609441 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:23.609441 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:23.609441 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:23.609441 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:23.609441 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:23.609441 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:23.609441 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:23.609441 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:23.609441 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:41:23.609441 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:41:23.609441 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:41:23.614305 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:41:23.614305 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:23.614305 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:23.614305 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:23.614305 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:23.614305 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:23.614305 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:23.614305 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:23.614305 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:23.614305 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:23.614305 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:23.614305 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:23.614305 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:23.614305 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:23.614305 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:23.614305 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:23.614305 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:23.615821 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:41:23.615821 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:41:23.615821 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:23.615821 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:23.615821 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 432/864] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:23.618622 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33776966-f21a-46d6-ab3a-dd099690ea14 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:23.646876 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1fc2d-a2f3-402d-82b5-14a300dfd7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:23.647569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1fc2d-a2f3-402d-82b5-14a300dfd7af None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:23.653423 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1fc2d-a2f3-402d-82b5-14a300dfd7af None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:23.653423 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d1fc2d-a2f3-402d-82b5-14a300dfd7af None admin] PATH_INFO: `/v3/services` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:23.653423 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d1fc2d-a2f3-402d-82b5-14a300dfd7af None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:23.653423 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d1fc2d-a2f3-402d-82b5-14a300dfd7af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:23.653423 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d1fc2d-a2f3-402d-82b5-14a300dfd7af None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:23.662643 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 433/865] 10.222.0.40 () {56 vars in 1126 bytes} [Sun Jul 17 16:41:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:23.664327 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1fc2d-a2f3-402d-82b5-14a300dfd7af None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:23.681663 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:23.770295 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:23.799711 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33776966-f21a-46d6-ab3a-dd099690ea14 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:23.810064 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d1fc2d-a2f3-402d-82b5-14a300dfd7af None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:23.854440 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf3b5cd-26d6-4f92-a8c7-1ac1e369fd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:23.855938 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf3b5cd-26d6-4f92-a8c7-1ac1e369fd0c None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:23.856488 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf3b5cd-26d6-4f92-a8c7-1ac1e369fd0c None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:23.857122 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf3b5cd-26d6-4f92-a8c7-1ac1e369fd0c None admin] PATH_INFO: `/v3/services` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:23.858540 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a651e-875c-4eed-83b8-3764aaf6c646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:23.859329 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a651e-875c-4eed-83b8-3764aaf6c646 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:23.859546 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf3b5cd-26d6-4f92-a8c7-1ac1e369fd0c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:23.859819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a651e-875c-4eed-83b8-3764aaf6c646 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:23.860196 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a651e-875c-4eed-83b8-3764aaf6c646 None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:23.860356 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf3b5cd-26d6-4f92-a8c7-1ac1e369fd0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:23.861153 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632a651e-875c-4eed-83b8-3764aaf6c646 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:23.861557 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632a651e-875c-4eed-83b8-3764aaf6c646 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:23.862786 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf3b5cd-26d6-4f92-a8c7-1ac1e369fd0c None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:23.863170 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632a651e-875c-4eed-83b8-3764aaf6c646 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:23.870880 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 434/866] 10.222.0.40 () {56 vars in 1126 bytes} [Sun Jul 17 16:41:23 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:41:23.880353 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 433/867] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:41:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:23.887073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf3b5cd-26d6-4f92-a8c7-1ac1e369fd0c None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:23.982603 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:24.014555 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf3b5cd-26d6-4f92-a8c7-1ac1e369fd0c None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:24.057412 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333ea57-66d3-46b1-82fc-067c5ce78ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:24.058330 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333ea57-66d3-46b1-82fc-067c5ce78ea9 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:24.058686 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333ea57-66d3-46b1-82fc-067c5ce78ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:24.059036 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333ea57-66d3-46b1-82fc-067c5ce78ea9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:24.068040 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333ea57-66d3-46b1-82fc-067c5ce78ea9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:24.069623 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333ea57-66d3-46b1-82fc-067c5ce78ea9 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:24.085901 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-7333ea57-66d3-46b1-82fc-067c5ce78ea9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:41:24.085901 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:24.085901 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:24.085901 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:24.085901 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:24.085901 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:24.085901 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:24.085901 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:24.085901 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:24.085901 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:24.085901 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:24.085901 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:24.085901 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:41:24.085901 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:41:24.085901 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:41:24.087677 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:41:24.087677 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:24.087677 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:24.087677 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:24.087677 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:24.087677 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:24.087677 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:24.087677 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:24.087677 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:24.087677 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:24.087677 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:24.087677 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:24.087677 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:24.087677 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:24.087677 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:24.087677 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:24.087677 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:24.089403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:41:24.089403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:41:24.089403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:41:24.089403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:41:24.089403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:41:24.089403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:41:24.089403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:41:24.089403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:24.089403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 435/868] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:41:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:24.094703 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a651e-875c-4eed-83b8-3764aaf6c646 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:24.223065 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a651e-875c-4eed-83b8-3764aaf6c646 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:24.260670 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d380593-dec4-4b31-8f62-177ab1310beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:24.261374 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d380593-dec4-4b31-8f62-177ab1310beb None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:24.261662 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d380593-dec4-4b31-8f62-177ab1310beb None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:24.261945 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d380593-dec4-4b31-8f62-177ab1310beb None admin] PATH_INFO: `/v3/projects` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:24.262821 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d380593-dec4-4b31-8f62-177ab1310beb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:24.263177 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d380593-dec4-4b31-8f62-177ab1310beb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:24.265811 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d380593-dec4-4b31-8f62-177ab1310beb None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:24.276506 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:24.281295 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 434/869] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:41:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:24.286984 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333ea57-66d3-46b1-82fc-067c5ce78ea9 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:24.399019 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333ea57-66d3-46b1-82fc-067c5ce78ea9 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:24.437987 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a0ef23-59ef-44f9-9fae-4b54b517ec62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:24.438577 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a0ef23-59ef-44f9-9fae-4b54b517ec62 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:24.438865 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a0ef23-59ef-44f9-9fae-4b54b517ec62 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:24.439146 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a0ef23-59ef-44f9-9fae-4b54b517ec62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:24.440003 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a0ef23-59ef-44f9-9fae-4b54b517ec62 None admin] RBAC: Adding query filter params (role.id=1025051975f64f689effa58dc4259e36, scope.project.id=927057f9a5af4833aeac9992203a223c, user.id=69482eeb1e8547aabca9f036b5c34f13) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:24.440323 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a0ef23-59ef-44f9-9fae-4b54b517ec62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:24.441813 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a0ef23-59ef-44f9-9fae-4b54b517ec62 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:24.444146 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:24.449201 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 436/870] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:41:24 2022] GET /identity/v3/role_assignments?user.id=69482eeb1e8547aabca9f036b5c34f13&scope.project.id=927057f9a5af4833aeac9992203a223c&role.id=1025051975f64f689effa58dc4259e36 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:24.706591 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4092c-0f60-47cd-b6bb-3479a19e816f None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:24.707091 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4092c-0f60-47cd-b6bb-3479a19e816f None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:24.707091 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4092c-0f60-47cd-b6bb-3479a19e816f None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:24.708799 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 435/871] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:24.714625 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09e7a2-b06d-4efa-a247-985fc952b897 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:24.714945 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09e7a2-b06d-4efa-a247-985fc952b897 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:24.715252 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09e7a2-b06d-4efa-a247-985fc952b897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:24.745759 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-3c09e7a2-b06d-4efa-a247-985fc952b897 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:24.747714 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-3c09e7a2-b06d-4efa-a247-985fc952b897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:24.877854 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 437/872] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:24 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:41:24.900843 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e4092c-0f60-47cd-b6bb-3479a19e816f None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:25.050362 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e4092c-0f60-47cd-b6bb-3479a19e816f None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:25.089404 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20f27e1-7d5b-46e2-9faa-a3b242350910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:25.090079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f27e1-7d5b-46e2-9faa-a3b242350910 None admin] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:25.090364 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f27e1-7d5b-46e2-9faa-a3b242350910 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:25.090682 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f27e1-7d5b-46e2-9faa-a3b242350910 None admin] PATH_INFO: `/v3/services` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:25.091386 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20f27e1-7d5b-46e2-9faa-a3b242350910 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:25.093794 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20f27e1-7d5b-46e2-9faa-a3b242350910 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:25.185022 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 436/873] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Jul 17 16:41:24 2022] POST /identity/v3/services => generated 251 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:41:26.265484 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd9d41-cdfb-4f02-8d10-f5eee3a8b2cf None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:26.266480 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd9d41-cdfb-4f02-8d10-f5eee3a8b2cf None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:26.266985 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd9d41-cdfb-4f02-8d10-f5eee3a8b2cf None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:26.268970 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 438/874] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:26.277390 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f66bd7-02af-4101-b6c8-744c1eb506f8 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:26.277898 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f66bd7-02af-4101-b6c8-744c1eb506f8 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:26.278322 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f66bd7-02af-4101-b6c8-744c1eb506f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:26.306349 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-58f66bd7-02af-4101-b6c8-744c1eb506f8 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:26.308317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-58f66bd7-02af-4101-b6c8-744c1eb506f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:26.415605 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:26.446701 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 437/875] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:26 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:41:26.486335 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd9d41-cdfb-4f02-8d10-f5eee3a8b2cf None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:26.642822 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd9d41-cdfb-4f02-8d10-f5eee3a8b2cf None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:26.674412 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:26.682918 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c0945-49ff-47e1-91a6-9baeda470ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:26.686228 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c0945-49ff-47e1-91a6-9baeda470ae2 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:26.686228 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c0945-49ff-47e1-91a6-9baeda470ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:26.686228 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c0945-49ff-47e1-91a6-9baeda470ae2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:26.697168 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c0945-49ff-47e1-91a6-9baeda470ae2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:41:26.706108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-1c0c0945-49ff-47e1-91a6-9baeda470ae2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 17 16:41:26.706108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:26.706108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:26.706108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:26.706108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:26.706108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:26.706108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:26.706108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:26.706108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:26.706108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:26.706108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:26.706108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:26.706108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:41:26.706108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:41:26.706108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:41:26.706108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:41:26.707087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:41:26.707087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:41:26.707087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:41:26.707087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:41:26.707087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:26.707087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:26.707087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:26.707087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:26.707087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:26.707087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:26.707087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:26.707087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:26.707087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:26.707087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:26.707087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:26.707087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:26.707940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:26.707940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:26.707940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:26.707940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:26.707940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:41:26.707940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:41:26.707940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:41:26.707940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:41:26.707940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:41:26.707940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:41:26.707940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 17 16:41:26.707940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:26.708580 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 439/876] 10.222.0.40 () {56 vars in 1120 bytes} [Sun Jul 17 16:41:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:26.717359 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f66bd7-02af-4101-b6c8-744c1eb506f8 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:26.785014 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc2b9b-932f-4d95-bace-c3c3cbf1b67b None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:26.785619 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc2b9b-932f-4d95-bace-c3c3cbf1b67b None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:26.786135 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc2b9b-932f-4d95-bace-c3c3cbf1b67b None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:26.790760 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 440/877] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:26 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:41:26.798049 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d29864-d7f1-4b7a-964b-1aafc52c3093 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:26.798367 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d29864-d7f1-4b7a-964b-1aafc52c3093 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:26.798601 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d29864-d7f1-4b7a-964b-1aafc52c3093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:26.817788 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-68d29864-d7f1-4b7a-964b-1aafc52c3093 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:26.819001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-68d29864-d7f1-4b7a-964b-1aafc52c3093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:26.861277 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f66bd7-02af-4101-b6c8-744c1eb506f8 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:26.901547 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5baea-a541-43e5-a69c-d051fed2c3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:26.902296 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5baea-a541-43e5-a69c-d051fed2c3dc None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:26.902585 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5baea-a541-43e5-a69c-d051fed2c3dc None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:26.902864 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5baea-a541-43e5-a69c-d051fed2c3dc None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:26.903628 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f5baea-a541-43e5-a69c-d051fed2c3dc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:26.903939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f5baea-a541-43e5-a69c-d051fed2c3dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:26.908945 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f5baea-a541-43e5-a69c-d051fed2c3dc None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:26.912842 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 441/878] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:26 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:41:26.912842 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448f985-b9c6-4d1d-a1b8-47113f5f0d7e None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:26.912842 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448f985-b9c6-4d1d-a1b8-47113f5f0d7e None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:26.913185 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448f985-b9c6-4d1d-a1b8-47113f5f0d7e None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:26.916868 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 442/879] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:26.920521 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65315815-a1d1-4ab5-98c6-ecb3d0e507b4 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:26.921196 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65315815-a1d1-4ab5-98c6-ecb3d0e507b4 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:26.921888 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65315815-a1d1-4ab5-98c6-ecb3d0e507b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:26.926596 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 438/880] 10.222.0.40 () {56 vars in 1115 bytes} [Sun Jul 17 16:41:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:26.932994 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5baea-a541-43e5-a69c-d051fed2c3dc None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:26.947534 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-65315815-a1d1-4ab5-98c6-ecb3d0e507b4 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:26.948671 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-65315815-a1d1-4ab5-98c6-ecb3d0e507b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:26.990315 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:27.029684 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 443/881] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:26 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 17 16:41:27.031422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65315815-a1d1-4ab5-98c6-ecb3d0e507b4 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:27.050923 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5baea-a541-43e5-a69c-d051fed2c3dc None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:27.089072 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c92d4f7-d4a8-4e5d-bc5c-32f0f63d7144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:27.089987 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92d4f7-d4a8-4e5d-bc5c-32f0f63d7144 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:27.090275 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92d4f7-d4a8-4e5d-bc5c-32f0f63d7144 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:27.090556 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92d4f7-d4a8-4e5d-bc5c-32f0f63d7144 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:27.092008 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:27.104426 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c92d4f7-d4a8-4e5d-bc5c-32f0f63d7144 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:27.105973 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c92d4f7-d4a8-4e5d-bc5c-32f0f63d7144 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:27.125396 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-4c92d4f7-d4a8-4e5d-bc5c-32f0f63d7144 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:27.125396 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:27.125396 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:27.125396 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:27.125396 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:27.125396 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:27.125396 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:27.125396 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:27.125396 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:27.125396 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:27.125396 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:27.125396 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:27.125396 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:41:27.125396 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:41:27.125396 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:41:27.126395 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:41:27.126395 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:27.126395 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:27.126395 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:27.126395 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:27.126395 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:27.126395 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:27.126395 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:27.126395 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:27.126395 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:27.126395 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:27.126395 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:27.126395 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:27.126395 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:27.126395 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:27.126395 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:27.126395 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:27.127329 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:41:27.127329 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:41:27.127329 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:27.127329 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:27.127329 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 439/882] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:27.128139 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c92d4f7-d4a8-4e5d-bc5c-32f0f63d7144 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:27.159476 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65315815-a1d1-4ab5-98c6-ecb3d0e507b4 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:27.206295 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0d3b5f-313f-42b1-acbc-eb459be05a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:27.207376 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d3b5f-313f-42b1-acbc-eb459be05a37 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:27.207376 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d3b5f-313f-42b1-acbc-eb459be05a37 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:27.207578 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0d3b5f-313f-42b1-acbc-eb459be05a37 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:27.208225 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0d3b5f-313f-42b1-acbc-eb459be05a37 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:27.209706 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0d3b5f-313f-42b1-acbc-eb459be05a37 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:27.223075 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-9a0d3b5f-313f-42b1-acbc-eb459be05a37 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 17 16:41:27.223075 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:27.223075 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 17 16:41:27.223075 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 17 16:41:27.223075 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 17 16:41:27.223075 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 17 16:41:27.223075 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 17 16:41:27.223075 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:27.223075 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 17 16:41:27.223075 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:27.223075 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 17 16:41:27.223075 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:27.223075 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:27.223075 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:27.223075 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:27.223075 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:27.223075 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:27.224150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:27.224150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:27.224150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:27.224150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:27.224150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:27.224150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:27.224150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 17 16:41:27.224150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 17 16:41:27.224150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 17 16:41:27.224150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 17 16:41:27.224150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:27.224150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:27.224150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:27.224150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:27.224150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:27.224150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:27.225579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:27.225579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:27.225579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:27.225579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:27.225579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:27.225579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:27.225579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:27.225579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:27.225579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:27.225579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:27.225579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 17 16:41:27.225579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:27.225579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 17 16:41:27.225579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:27.225579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 444/883] 10.222.0.40 () {56 vars in 1126 bytes} [Sun Jul 17 16:41:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:27.226380 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0d3b5f-313f-42b1-acbc-eb459be05a37 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:27.249731 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:27.253131 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c92d4f7-d4a8-4e5d-bc5c-32f0f63d7144 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:27.299634 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2703caac-cd02-4d8d-86ab-6c288586e345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:27.301230 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703caac-cd02-4d8d-86ab-6c288586e345 None admin] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:27.302170 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703caac-cd02-4d8d-86ab-6c288586e345 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:27.302608 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703caac-cd02-4d8d-86ab-6c288586e345 None admin] PATH_INFO: `/v3/services` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:27.303762 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2703caac-cd02-4d8d-86ab-6c288586e345 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:27.306421 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2703caac-cd02-4d8d-86ab-6c288586e345 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:27.337428 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0d3b5f-313f-42b1-acbc-eb459be05a37 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:27.351351 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:27.372924 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06765dd9-b039-4fb2-ae8f-3425faff382f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:27.374390 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06765dd9-b039-4fb2-ae8f-3425faff382f None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:27.374754 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06765dd9-b039-4fb2-ae8f-3425faff382f None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:27.375033 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06765dd9-b039-4fb2-ae8f-3425faff382f None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:27.375763 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06765dd9-b039-4fb2-ae8f-3425faff382f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:27.376075 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06765dd9-b039-4fb2-ae8f-3425faff382f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:27.377489 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06765dd9-b039-4fb2-ae8f-3425faff382f None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:27.390082 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 445/884] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:27.390940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06765dd9-b039-4fb2-ae8f-3425faff382f None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:27.404031 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 440/885] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Jul 17 16:41:27 2022] POST /identity/v3/services => generated 269 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:41:27.405477 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee6278-0033-4474-80df-7f3db8af9666 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:27.405774 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee6278-0033-4474-80df-7f3db8af9666 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:27.406057 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee6278-0033-4474-80df-7f3db8af9666 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:27.407606 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 441/886] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:27.412094 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee6278-0033-4474-80df-7f3db8af9666 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:27.531167 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06765dd9-b039-4fb2-ae8f-3425faff382f None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:27.547132 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee6278-0033-4474-80df-7f3db8af9666 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:27.581070 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc74b06b-99bb-480f-be5b-45ef062b3cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:27.581759 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74b06b-99bb-480f-be5b-45ef062b3cd1 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:27.582056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74b06b-99bb-480f-be5b-45ef062b3cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:27.582343 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74b06b-99bb-480f-be5b-45ef062b3cd1 None admin] PATH_INFO: `/v3/services` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:27.583076 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc74b06b-99bb-480f-be5b-45ef062b3cd1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:27.583399 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc74b06b-99bb-480f-be5b-45ef062b3cd1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:27.584833 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc74b06b-99bb-480f-be5b-45ef062b3cd1 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:27.591038 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd1859-8796-4623-aee0-5fb5c928d1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:27.592446 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd1859-8796-4623-aee0-5fb5c928d1ad None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:27.592738 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd1859-8796-4623-aee0-5fb5c928d1ad None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:27.593042 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd1859-8796-4623-aee0-5fb5c928d1ad None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:27.594182 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 446/887] 10.222.0.40 () {56 vars in 1122 bytes} [Sun Jul 17 16:41:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:27.595446 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2411a5b-46cd-4eef-8ef5-5ea5ebeffbb4 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:27.595791 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2411a5b-46cd-4eef-8ef5-5ea5ebeffbb4 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:27.596108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2411a5b-46cd-4eef-8ef5-5ea5ebeffbb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:27.602819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd1859-8796-4623-aee0-5fb5c928d1ad None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:27.604283 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd1859-8796-4623-aee0-5fb5c928d1ad None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:27.615101 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-e2411a5b-46cd-4eef-8ef5-5ea5ebeffbb4 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:27.616338 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-e2411a5b-46cd-4eef-8ef5-5ea5ebeffbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:27.623793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-9dfd1859-8796-4623-aee0-5fb5c928d1ad None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 17 16:41:27.623793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:27.623793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:27.623793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:27.623793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:27.623793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:27.623793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:27.623793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:27.623793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:27.623793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:27.623793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:27.623793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:27.623793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:41:27.623793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:41:27.623793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:41:27.623793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:41:27.625250 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:27.625250 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:27.625250 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:41:27.625250 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:41:27.625250 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:41:27.625250 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:41:27.625250 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:27.625250 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:27.625250 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:27.625250 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:27.625250 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:27.625250 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:27.625250 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:27.625250 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:27.625250 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:27.625250 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:27.625250 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:27.625250 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:27.626443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:27.626443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:27.626443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:41:27.626443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:41:27.626443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:41:27.626443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:41:27.626443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:41:27.626443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:41:27.626443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 17 16:41:27.626443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:27.626443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 442/888] 10.222.0.40 () {56 vars in 1117 bytes} [Sun Jul 17 16:41:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:27.626443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd1859-8796-4623-aee0-5fb5c928d1ad None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:27.646639 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:27.653519 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:27.697005 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 447/889] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:27 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:41:27.698007 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2411a5b-46cd-4eef-8ef5-5ea5ebeffbb4 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:27.758158 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd1859-8796-4623-aee0-5fb5c928d1ad None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:27.799913 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d33bc50-ff51-4731-bf21-f6166573e43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:27.800511 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33bc50-ff51-4731-bf21-f6166573e43e None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:27.800809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33bc50-ff51-4731-bf21-f6166573e43e None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:27.801101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33bc50-ff51-4731-bf21-f6166573e43e None admin] PATH_INFO: `/v3/services` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:27.801827 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d33bc50-ff51-4731-bf21-f6166573e43e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:27.802136 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d33bc50-ff51-4731-bf21-f6166573e43e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:27.803529 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d33bc50-ff51-4731-bf21-f6166573e43e None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:27.808723 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 443/890] 10.222.0.40 () {56 vars in 1122 bytes} [Sun Jul 17 16:41:27 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:27.809616 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d33bc50-ff51-4731-bf21-f6166573e43e None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:27.820369 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2411a5b-46cd-4eef-8ef5-5ea5ebeffbb4 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:27.870782 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f7ae12-52e6-4879-8190-620fc463f567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:27.871930 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7ae12-52e6-4879-8190-620fc463f567 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:27.872445 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7ae12-52e6-4879-8190-620fc463f567 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:27.873521 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7ae12-52e6-4879-8190-620fc463f567 None admin] PATH_INFO: `/v3/users` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:27.874646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f7ae12-52e6-4879-8190-620fc463f567 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:27.875221 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f7ae12-52e6-4879-8190-620fc463f567 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:27.877712 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f7ae12-52e6-4879-8190-620fc463f567 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:27.906935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 448/891] 10.222.0.40 () {56 vars in 1149 bytes} [Sun Jul 17 16:41:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:27.912923 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f7ae12-52e6-4879-8190-620fc463f567 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:27.939041 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:27.952233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:27.964587 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d33bc50-ff51-4731-bf21-f6166573e43e None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:28.016562 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28885947-0e16-4c61-a77c-4f73fab2a904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:28.017422 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28885947-0e16-4c61-a77c-4f73fab2a904 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:28.017709 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28885947-0e16-4c61-a77c-4f73fab2a904 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:28.017990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28885947-0e16-4c61-a77c-4f73fab2a904 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:28.018722 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28885947-0e16-4c61-a77c-4f73fab2a904 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:28.020308 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28885947-0e16-4c61-a77c-4f73fab2a904 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:28.035010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-28885947-0e16-4c61-a77c-4f73fab2a904 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 17 16:41:28.035010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:28.035010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 17 16:41:28.035010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 17 16:41:28.035010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 17 16:41:28.035010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 17 16:41:28.035010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 17 16:41:28.035010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:28.035010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 17 16:41:28.035010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:28.035010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 17 16:41:28.035010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:28.035010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:28.035010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:28.035010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:28.035010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:28.035010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:28.036051 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:28.036051 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:28.036051 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:28.036051 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:28.036051 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:28.036051 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:28.036051 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 17 16:41:28.036051 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 17 16:41:28.036051 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 17 16:41:28.036051 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 17 16:41:28.036051 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:28.036051 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:28.036051 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:28.036051 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:28.036051 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:28.036051 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:28.036890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:28.036890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:28.036890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:28.036890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:28.036890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:28.036890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:28.036890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:28.036890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:28.036890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:28.036890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:28.036890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 17 16:41:28.036890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:28.036890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 17 16:41:28.036890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:28.039386 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 444/892] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 17 16:41:27 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:28.045877 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28885947-0e16-4c61-a77c-4f73fab2a904 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:28.062016 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f7ae12-52e6-4879-8190-620fc463f567 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:28.106467 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d1b01-f969-47bb-9ff6-8928ff8b785e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:28.109114 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d1b01-f969-47bb-9ff6-8928ff8b785e None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:28.109661 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d1b01-f969-47bb-9ff6-8928ff8b785e None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:28.110080 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d1b01-f969-47bb-9ff6-8928ff8b785e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:28.172006 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28885947-0e16-4c61-a77c-4f73fab2a904 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:28.218358 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d1b01-f969-47bb-9ff6-8928ff8b785e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:28.219654 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b3d5e5-3f16-406e-b997-8663c1be0a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:28.219869 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d1b01-f969-47bb-9ff6-8928ff8b785e None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:28.220770 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b3d5e5-3f16-406e-b997-8663c1be0a5a None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:28.221189 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b3d5e5-3f16-406e-b997-8663c1be0a5a None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:28.221566 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b3d5e5-3f16-406e-b997-8663c1be0a5a None admin] PATH_INFO: `/v3/services` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:28.222448 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b3d5e5-3f16-406e-b997-8663c1be0a5a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:28.222856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b3d5e5-3f16-406e-b997-8663c1be0a5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:28.224386 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b3d5e5-3f16-406e-b997-8663c1be0a5a None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:28.236106 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 445/893] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 17 16:41:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:41:28.237342 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b3d5e5-3f16-406e-b997-8663c1be0a5a None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:28.250502 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:28.251475 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:28.343935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 449/894] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:41:27 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 17 16:41:28.349883 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d1b01-f969-47bb-9ff6-8928ff8b785e None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:28.363614 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b3d5e5-3f16-406e-b997-8663c1be0a5a None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:28.402801 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82604f7a-6b69-4f26-947e-67c02ec0a264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:28.403602 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82604f7a-6b69-4f26-947e-67c02ec0a264 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:28.403980 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82604f7a-6b69-4f26-947e-67c02ec0a264 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:28.405078 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82604f7a-6b69-4f26-947e-67c02ec0a264 None admin] PATH_INFO: `/v3/services` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:28.406007 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82604f7a-6b69-4f26-947e-67c02ec0a264 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:28.406389 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82604f7a-6b69-4f26-947e-67c02ec0a264 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:28.407952 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82604f7a-6b69-4f26-947e-67c02ec0a264 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:28.413791 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 446/895] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 17 16:41:28 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:28.419566 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82604f7a-6b69-4f26-947e-67c02ec0a264 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:28.489673 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d1b01-f969-47bb-9ff6-8928ff8b785e None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:28.510961 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:28.527018 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:28.540007 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a912fbfc-90c8-48a7-9492-0301738e7746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:28.541031 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912fbfc-90c8-48a7-9492-0301738e7746 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:28.541417 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912fbfc-90c8-48a7-9492-0301738e7746 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:28.541795 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912fbfc-90c8-48a7-9492-0301738e7746 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:28.551498 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82604f7a-6b69-4f26-947e-67c02ec0a264 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:28.554779 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a912fbfc-90c8-48a7-9492-0301738e7746 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:28.556473 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a912fbfc-90c8-48a7-9492-0301738e7746 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:28.581270 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-a912fbfc-90c8-48a7-9492-0301738e7746 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:28.581270 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:28.581270 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:28.581270 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:28.581270 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:28.581270 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:28.581270 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:28.581270 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:28.581270 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:28.581270 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:28.581270 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:28.581270 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:28.581270 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:41:28.581270 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:41:28.581270 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:41:28.582475 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:41:28.582475 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:28.582475 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:28.582475 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:28.582475 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:28.582475 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:28.582475 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:28.582475 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:28.582475 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:28.582475 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:28.582475 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:28.582475 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:28.582475 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:28.582475 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:28.582475 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:28.582475 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:28.582475 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:28.583534 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:41:28.583534 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:41:28.583534 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:28.583534 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:28.590729 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 450/896] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:28.594930 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a912fbfc-90c8-48a7-9492-0301738e7746 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:28.602362 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df0293-857a-46bb-951a-83e6f97ced34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:28.603046 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df0293-857a-46bb-951a-83e6f97ced34 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:28.604983 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df0293-857a-46bb-951a-83e6f97ced34 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:28.604983 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df0293-857a-46bb-951a-83e6f97ced34 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:28.604983 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df0293-857a-46bb-951a-83e6f97ced34 None admin] RBAC: Adding query filter params (interface=public, service_id=fa9acea906f748bdb20fa348ceb7490f, region_id=RegionOne) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:28.604983 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df0293-857a-46bb-951a-83e6f97ced34 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:28.606318 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df0293-857a-46bb-951a-83e6f97ced34 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:28.618372 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 447/897] 10.222.0.40 () {56 vars in 1251 bytes} [Sun Jul 17 16:41:28 2022] GET /identity/v3/endpoints?service_id=fa9acea906f748bdb20fa348ceb7490f&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:41:28.619480 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df0293-857a-46bb-951a-83e6f97ced34 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:28.723503 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a912fbfc-90c8-48a7-9492-0301738e7746 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:28.749130 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df0293-857a-46bb-951a-83e6f97ced34 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:28.764268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1708fd4-0e35-43f6-80bb-143855fd583d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:28.765315 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1708fd4-0e35-43f6-80bb-143855fd583d None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:28.765713 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1708fd4-0e35-43f6-80bb-143855fd583d None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:28.766001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1708fd4-0e35-43f6-80bb-143855fd583d None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:28.766975 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1708fd4-0e35-43f6-80bb-143855fd583d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:28.767290 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1708fd4-0e35-43f6-80bb-143855fd583d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:28.768742 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1708fd4-0e35-43f6-80bb-143855fd583d None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:28.784342 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 451/898] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:41:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:28.790174 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1708fd4-0e35-43f6-80bb-143855fd583d None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:28.790460 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0cb133-3f9d-4664-8780-b603f3eeb2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:28.791226 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0cb133-3f9d-4664-8780-b603f3eeb2d4 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:28.791579 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0cb133-3f9d-4664-8780-b603f3eeb2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:28.791871 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0cb133-3f9d-4664-8780-b603f3eeb2d4 None admin] PATH_INFO: `/v3/services` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:28.792622 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0cb133-3f9d-4664-8780-b603f3eeb2d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:28.794148 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0cb133-3f9d-4664-8780-b603f3eeb2d4 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:28.801721 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 448/899] 10.222.0.40 () {56 vars in 1087 bytes} [Sun Jul 17 16:41:28 2022] GET /identity/v3/services => generated 1268 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 17 16:41:28.804550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:28.836585 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:28.913987 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1708fd4-0e35-43f6-80bb-143855fd583d None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:28.970472 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4499d740-412d-4410-b8d6-dded0755a232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:28.971317 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499d740-412d-4410-b8d6-dded0755a232 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:28.971599 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499d740-412d-4410-b8d6-dded0755a232 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:28.971872 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499d740-412d-4410-b8d6-dded0755a232 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:28.980392 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4499d740-412d-4410-b8d6-dded0755a232 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:28.981942 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4499d740-412d-4410-b8d6-dded0755a232 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:28.998917 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-4499d740-412d-4410-b8d6-dded0755a232 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:41:28.998917 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:28.998917 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:28.998917 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:28.998917 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:28.998917 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:28.998917 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:28.998917 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:28.998917 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:28.998917 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:28.998917 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:28.998917 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:28.998917 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:41:28.998917 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:41:28.998917 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:41:28.999911 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:41:28.999911 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:28.999911 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:28.999911 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:28.999911 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:28.999911 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:28.999911 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:28.999911 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:28.999911 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:28.999911 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:28.999911 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:28.999911 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:28.999911 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:28.999911 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:28.999911 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:28.999911 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:28.999911 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:29.000820 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:41:29.000820 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:41:29.000820 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:41:29.000820 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:41:29.000820 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:41:29.000820 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:41:29.000820 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:41:29.000820 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:29.001634 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 452/900] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:41:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:29.010169 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0cb133-3f9d-4664-8780-b603f3eeb2d4 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:29.214030 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0cb133-3f9d-4664-8780-b603f3eeb2d4 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:29.280050 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f902cf5-8a85-492c-839e-cd23cb8db512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:29.281118 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f902cf5-8a85-492c-839e-cd23cb8db512 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:29.281566 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f902cf5-8a85-492c-839e-cd23cb8db512 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:29.282067 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f902cf5-8a85-492c-839e-cd23cb8db512 None admin] PATH_INFO: `/v3/projects` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:29.283605 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f902cf5-8a85-492c-839e-cd23cb8db512 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:29.284026 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f902cf5-8a85-492c-839e-cd23cb8db512 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:29.286703 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f902cf5-8a85-492c-839e-cd23cb8db512 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:29.309731 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 449/901] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:41:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:29.319823 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4499d740-412d-4410-b8d6-dded0755a232 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:29.485835 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:29.492658 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4499d740-412d-4410-b8d6-dded0755a232 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:29.532722 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2e77e2-3374-4885-a8fd-618b0b4a9069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:29.533428 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e77e2-3374-4885-a8fd-618b0b4a9069 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:29.533709 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e77e2-3374-4885-a8fd-618b0b4a9069 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:29.533985 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e77e2-3374-4885-a8fd-618b0b4a9069 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:29.534831 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2e77e2-3374-4885-a8fd-618b0b4a9069 None admin] RBAC: Adding query filter params (role.id=68d3a95a9fc34fd1a28a9861d22ee828, scope.project.id=927057f9a5af4833aeac9992203a223c, user.id=37ed5f72dc8c4a5685bd1245cadcdf7e) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:29.535144 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2e77e2-3374-4885-a8fd-618b0b4a9069 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:29.536594 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2e77e2-3374-4885-a8fd-618b0b4a9069 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:29.543579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 453/902] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:41:29 2022] GET /identity/v3/role_assignments?user.id=37ed5f72dc8c4a5685bd1245cadcdf7e&scope.project.id=927057f9a5af4833aeac9992203a223c&role.id=68d3a95a9fc34fd1a28a9861d22ee828 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:29.669728 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a921f6-52da-4897-85c4-b23e968566b1 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:29.670755 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a921f6-52da-4897-85c4-b23e968566b1 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:29.671257 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a921f6-52da-4897-85c4-b23e968566b1 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:29.673421 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 450/903] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:29.680057 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010c538-9dac-483a-b2f0-a8225f31c717 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:29.680707 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010c538-9dac-483a-b2f0-a8225f31c717 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:29.681272 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010c538-9dac-483a-b2f0-a8225f31c717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:29.700742 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-5010c538-9dac-483a-b2f0-a8225f31c717 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:29.703371 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-5010c538-9dac-483a-b2f0-a8225f31c717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:29.805174 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 454/904] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:29 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:41:29.829693 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a921f6-52da-4897-85c4-b23e968566b1 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:29.912054 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:29.977386 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a921f6-52da-4897-85c4-b23e968566b1 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:30.044508 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc1a94c-c17a-4c89-8bc5-4fdb2fb39249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:30.046299 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc1a94c-c17a-4c89-8bc5-4fdb2fb39249 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:30.046758 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc1a94c-c17a-4c89-8bc5-4fdb2fb39249 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:30.047223 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc1a94c-c17a-4c89-8bc5-4fdb2fb39249 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:30.048377 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc1a94c-c17a-4c89-8bc5-4fdb2fb39249 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:30.050902 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc1a94c-c17a-4c89-8bc5-4fdb2fb39249 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:30.075631 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-4cc1a94c-c17a-4c89-8bc5-4fdb2fb39249 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 17 16:41:30.075631 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:30.075631 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 17 16:41:30.075631 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 17 16:41:30.075631 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 17 16:41:30.075631 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 17 16:41:30.075631 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 17 16:41:30.075631 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:30.075631 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 17 16:41:30.075631 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:30.075631 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 17 16:41:30.075631 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:30.075631 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:30.075631 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:30.075631 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:30.075631 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:30.077769 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:30.077769 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:30.077769 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:30.077769 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:30.077769 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:30.077769 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:30.077769 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:30.077769 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 17 16:41:30.077769 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 17 16:41:30.077769 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 17 16:41:30.077769 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 17 16:41:30.077769 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:30.077769 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:30.077769 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:30.077769 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:30.077769 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:30.078624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:30.078624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:30.078624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:30.078624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:30.078624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:30.078624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:30.078624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:30.078624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:30.078624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:30.078624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:30.078624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:30.078624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 17 16:41:30.078624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:30.078624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 17 16:41:30.078624 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:30.079417 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 451/905] 10.222.0.40 () {56 vars in 1132 bytes} [Sun Jul 17 16:41:29 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:30.084672 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5010c538-9dac-483a-b2f0-a8225f31c717 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:30.212623 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:30.223890 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5010c538-9dac-483a-b2f0-a8225f31c717 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:30.263453 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b93c55-8f68-46b2-8a70-38d1e758b29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:30.264678 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b93c55-8f68-46b2-8a70-38d1e758b29b None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:30.264997 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b93c55-8f68-46b2-8a70-38d1e758b29b None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:30.265252 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b93c55-8f68-46b2-8a70-38d1e758b29b None admin] PATH_INFO: `/v3/services` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:30.265980 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b93c55-8f68-46b2-8a70-38d1e758b29b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:30.266293 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b93c55-8f68-46b2-8a70-38d1e758b29b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:30.267691 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b93c55-8f68-46b2-8a70-38d1e758b29b None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:30.274625 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 455/906] 10.222.0.40 () {56 vars in 1126 bytes} [Sun Jul 17 16:41:30 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:30.277382 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5b1e0-fe5c-428d-8a1c-e9118b8aa438 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:30.277957 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5b1e0-fe5c-428d-8a1c-e9118b8aa438 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:30.278480 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5b1e0-fe5c-428d-8a1c-e9118b8aa438 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:30.279830 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b93c55-8f68-46b2-8a70-38d1e758b29b None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:30.281602 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 452/907] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:30.290927 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e13b6-5567-4e31-96f0-b92e9da277d4 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:30.291435 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e13b6-5567-4e31-96f0-b92e9da277d4 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:30.291948 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e13b6-5567-4e31-96f0-b92e9da277d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:30.325331 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-f71e13b6-5567-4e31-96f0-b92e9da277d4 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:30.330157 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-f71e13b6-5567-4e31-96f0-b92e9da277d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:30.476077 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b93c55-8f68-46b2-8a70-38d1e758b29b None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:30.481344 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 453/908] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:30 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 17 16:41:30.520097 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71e13b6-5567-4e31-96f0-b92e9da277d4 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:30.552708 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd7324a-f0be-48c5-946a-ecbbd8a06e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:30.554129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7324a-f0be-48c5-946a-ecbbd8a06e9c None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:30.554691 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7324a-f0be-48c5-946a-ecbbd8a06e9c None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:30.555184 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd7324a-f0be-48c5-946a-ecbbd8a06e9c None admin] PATH_INFO: `/v3/services` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:30.557054 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd7324a-f0be-48c5-946a-ecbbd8a06e9c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:30.557577 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd7324a-f0be-48c5-946a-ecbbd8a06e9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:30.560645 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd7324a-f0be-48c5-946a-ecbbd8a06e9c None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:30.576928 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 456/909] 10.222.0.40 () {56 vars in 1126 bytes} [Sun Jul 17 16:41:30 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:41:30.583509 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd7324a-f0be-48c5-946a-ecbbd8a06e9c None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:30.627614 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:30.689271 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:30.735945 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71e13b6-5567-4e31-96f0-b92e9da277d4 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:30.807205 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921b79d-f7b2-4d11-9b34-ca5df59d9fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:30.808950 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921b79d-f7b2-4d11-9b34-ca5df59d9fcb None admin] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:30.809643 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921b79d-f7b2-4d11-9b34-ca5df59d9fcb None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:30.810766 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921b79d-f7b2-4d11-9b34-ca5df59d9fcb None admin] PATH_INFO: `/v3/services` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:30.812520 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a921b79d-f7b2-4d11-9b34-ca5df59d9fcb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:30.815018 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a921b79d-f7b2-4d11-9b34-ca5df59d9fcb None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:30.817546 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd7324a-f0be-48c5-946a-ecbbd8a06e9c None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:30.886675 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eacafdc-d51a-4f2d-ba76-61ed130b7357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:30.887698 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eacafdc-d51a-4f2d-ba76-61ed130b7357 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:30.888188 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eacafdc-d51a-4f2d-ba76-61ed130b7357 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:30.889376 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eacafdc-d51a-4f2d-ba76-61ed130b7357 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:30.891170 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eacafdc-d51a-4f2d-ba76-61ed130b7357 None admin] RBAC: Adding query filter params (interface=public, service_id=8e017cfe40534249a088dd1b14c986ea, region_id=RegionOne) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:30.891736 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eacafdc-d51a-4f2d-ba76-61ed130b7357 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:30.894375 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eacafdc-d51a-4f2d-ba76-61ed130b7357 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:30.908252 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 454/910] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Jul 17 16:41:30 2022] POST /identity/v3/services => generated 241 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:41:30.912886 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 457/911] 10.222.0.40 () {56 vars in 1251 bytes} [Sun Jul 17 16:41:30 2022] GET /identity/v3/endpoints?service_id=8e017cfe40534249a088dd1b14c986ea&interface=public®ion_id=RegionOne => generated 194 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:41:30.921740 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921b79d-f7b2-4d11-9b34-ca5df59d9fcb None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:31.135037 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921b79d-f7b2-4d11-9b34-ca5df59d9fcb None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:31.165285 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:31.167895 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13928c07-ea5f-4ba8-90e3-15db300cdfe3 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:31.168483 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13928c07-ea5f-4ba8-90e3-15db300cdfe3 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:31.169051 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13928c07-ea5f-4ba8-90e3-15db300cdfe3 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:31.172229 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 458/912] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:41:31.180431 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c6723-c107-4c30-a4a5-682c6ebf70f4 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:31.181131 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c6723-c107-4c30-a4a5-682c6ebf70f4 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:31.181672 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c6723-c107-4c30-a4a5-682c6ebf70f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:31.206124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-a36c6723-c107-4c30-a4a5-682c6ebf70f4 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:31.207792 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-a36c6723-c107-4c30-a4a5-682c6ebf70f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:31.212949 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fbf8a5-53e8-4e15-bfc4-5b11998c348b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:31.214021 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fbf8a5-53e8-4e15-bfc4-5b11998c348b None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:31.214503 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fbf8a5-53e8-4e15-bfc4-5b11998c348b None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:31.214984 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fbf8a5-53e8-4e15-bfc4-5b11998c348b None admin] PATH_INFO: `/v3/services` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:31.216188 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fbf8a5-53e8-4e15-bfc4-5b11998c348b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:31.219290 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fbf8a5-53e8-4e15-bfc4-5b11998c348b None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:31.237642 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 455/913] 10.222.0.40 () {56 vars in 1087 bytes} [Sun Jul 17 16:41:30 2022] GET /identity/v3/services => generated 1497 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 17 16:41:31.324833 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 459/914] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:41:31.349477 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fbf8a5-53e8-4e15-bfc4-5b11998c348b None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:31.522365 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fbf8a5-53e8-4e15-bfc4-5b11998c348b None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:31.563265 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded8ed1-1173-4a26-a03f-754dd72a8b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:31.564086 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded8ed1-1173-4a26-a03f-754dd72a8b5c None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:31.564379 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded8ed1-1173-4a26-a03f-754dd72a8b5c None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:31.564661 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded8ed1-1173-4a26-a03f-754dd72a8b5c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:31.565367 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded8ed1-1173-4a26-a03f-754dd72a8b5c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:31.566760 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded8ed1-1173-4a26-a03f-754dd72a8b5c None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:31.579909 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-6ded8ed1-1173-4a26-a03f-754dd72a8b5c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 17 16:41:31.579909 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:31.579909 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 17 16:41:31.579909 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 17 16:41:31.579909 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 17 16:41:31.579909 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 17 16:41:31.579909 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 17 16:41:31.579909 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:31.579909 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 17 16:41:31.579909 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:31.579909 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 17 16:41:31.579909 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:31.579909 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:31.579909 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:31.579909 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:31.579909 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:31.579909 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:31.581816 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:31.581816 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:31.581816 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:31.581816 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:31.581816 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:31.581816 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:31.581816 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 17 16:41:31.581816 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 17 16:41:31.581816 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 17 16:41:31.581816 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 17 16:41:31.581816 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:31.581816 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:31.581816 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:31.581816 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:31.581816 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:31.581816 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:31.583402 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:31.583402 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:31.583402 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:31.583402 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:31.583402 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:31.583402 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:31.583402 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:31.583402 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:31.583402 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:31.583402 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:31.583402 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 17 16:41:31.583402 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:31.583402 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 17 16:41:31.583402 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:31.583402 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 456/915] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 17 16:41:31 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:31.588352 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c6723-c107-4c30-a4a5-682c6ebf70f4 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:31.617766 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:31.783819 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c6723-c107-4c30-a4a5-682c6ebf70f4 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:31.851162 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d2e51-1531-44de-84f6-eac116138d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:31.852333 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d2e51-1531-44de-84f6-eac116138d28 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:31.852837 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d2e51-1531-44de-84f6-eac116138d28 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:31.853330 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d2e51-1531-44de-84f6-eac116138d28 None admin] PATH_INFO: `/v3/services` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:31.854507 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90d2e51-1531-44de-84f6-eac116138d28 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:31.855031 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90d2e51-1531-44de-84f6-eac116138d28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:31.858819 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90d2e51-1531-44de-84f6-eac116138d28 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:31.868926 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 460/916] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 17 16:41:31 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:41:31.875122 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded8ed1-1173-4a26-a03f-754dd72a8b5c None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:31.921452 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:31.991728 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded8ed1-1173-4a26-a03f-754dd72a8b5c None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:32.035791 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2934af6-086f-47b1-b424-1bbc8ef52df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:32.036803 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2934af6-086f-47b1-b424-1bbc8ef52df8 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:32.037529 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2934af6-086f-47b1-b424-1bbc8ef52df8 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:32.038034 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2934af6-086f-47b1-b424-1bbc8ef52df8 None admin] PATH_INFO: `/v3/services` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:32.039340 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2934af6-086f-47b1-b424-1bbc8ef52df8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:32.039887 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2934af6-086f-47b1-b424-1bbc8ef52df8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:32.042443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2934af6-086f-47b1-b424-1bbc8ef52df8 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:32.053668 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 457/917] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 17 16:41:31 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:32.059608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d2e51-1531-44de-84f6-eac116138d28 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:32.181882 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:32.205961 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d2e51-1531-44de-84f6-eac116138d28 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:32.243827 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c71c3-6d56-4dcb-b54c-9f66451fdeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:32.244501 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c71c3-6d56-4dcb-b54c-9f66451fdeff None admin] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:32.244809 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c71c3-6d56-4dcb-b54c-9f66451fdeff None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:32.245084 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c71c3-6d56-4dcb-b54c-9f66451fdeff None admin] PATH_INFO: `/v3/endpoints` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:32.245834 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09c71c3-6d56-4dcb-b54c-9f66451fdeff None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:32.247216 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09c71c3-6d56-4dcb-b54c-9f66451fdeff None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:32.309197 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60000e7-8a4f-45ed-8d6d-4f921b026b67 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:32.309638 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60000e7-8a4f-45ed-8d6d-4f921b026b67 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:32.310491 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60000e7-8a4f-45ed-8d6d-4f921b026b67 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:32.313250 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 458/918] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:32.322404 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19145c6-d191-4f6e-bb3a-f46ba2b52a48 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:32.323045 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19145c6-d191-4f6e-bb3a-f46ba2b52a48 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:32.323656 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19145c6-d191-4f6e-bb3a-f46ba2b52a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:32.355321 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-f19145c6-d191-4f6e-bb3a-f46ba2b52a48 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:32.358058 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-f19145c6-d191-4f6e-bb3a-f46ba2b52a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:32.386329 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 461/919] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 17 16:41:32 2022] POST /identity/v3/endpoints => generated 346 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:41:32.506255 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 459/920] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:41:32.544581 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c71c3-6d56-4dcb-b54c-9f66451fdeff None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:32.740860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c71c3-6d56-4dcb-b54c-9f66451fdeff None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:32.796609 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:32.805456 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91a6a4-cc40-4b00-8592-17e5446d6d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:32.806837 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91a6a4-cc40-4b00-8592-17e5446d6d83 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:32.807448 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91a6a4-cc40-4b00-8592-17e5446d6d83 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:32.812925 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91a6a4-cc40-4b00-8592-17e5446d6d83 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:32.829225 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c91a6a4-cc40-4b00-8592-17e5446d6d83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:41:32.848501 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-2c91a6a4-cc40-4b00-8592-17e5446d6d83 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 17 16:41:32.848501 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:32.848501 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:32.848501 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:32.848501 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:32.848501 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:32.848501 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:32.848501 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:32.848501 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:32.848501 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:32.848501 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:32.848501 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:32.848501 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:41:32.848501 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:41:32.848501 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:41:32.848501 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:41:32.850305 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:41:32.850305 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:41:32.850305 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:41:32.850305 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:41:32.850305 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:32.850305 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:32.850305 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:32.850305 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:32.850305 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:32.850305 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:32.850305 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:32.850305 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:32.850305 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:32.850305 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:32.850305 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:32.850305 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:32.851562 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:32.851562 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:32.851562 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:32.851562 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:32.851562 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:41:32.851562 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:41:32.851562 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:41:32.851562 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:41:32.851562 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:41:32.851562 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:41:32.851562 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 17 16:41:32.851562 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:32.851562 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 462/921] 10.222.0.40 () {56 vars in 1120 bytes} [Sun Jul 17 16:41:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:32.857852 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19145c6-d191-4f6e-bb3a-f46ba2b52a48 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:32.916042 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:33.062850 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19145c6-d191-4f6e-bb3a-f46ba2b52a48 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:33.135857 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86baa892-2de1-4fe6-96da-370b507a7095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:33.137025 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86baa892-2de1-4fe6-96da-370b507a7095 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:33.137533 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86baa892-2de1-4fe6-96da-370b507a7095 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:33.138030 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86baa892-2de1-4fe6-96da-370b507a7095 None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:33.140078 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86baa892-2de1-4fe6-96da-370b507a7095 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:33.140636 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86baa892-2de1-4fe6-96da-370b507a7095 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:33.143169 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86baa892-2de1-4fe6-96da-370b507a7095 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:33.160082 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 460/922] 10.222.0.40 () {56 vars in 1115 bytes} [Sun Jul 17 16:41:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:33.171528 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91a6a4-cc40-4b00-8592-17e5446d6d83 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:33.296200 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91a6a4-cc40-4b00-8592-17e5446d6d83 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:33.335739 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff33b5a-c192-4d95-a06f-28322319db71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:33.336913 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff33b5a-c192-4d95-a06f-28322319db71 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:33.337422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff33b5a-c192-4d95-a06f-28322319db71 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:33.337895 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff33b5a-c192-4d95-a06f-28322319db71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:33.350789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff33b5a-c192-4d95-a06f-28322319db71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:33.352619 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff33b5a-c192-4d95-a06f-28322319db71 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:33.372379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-dff33b5a-c192-4d95-a06f-28322319db71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:33.372379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:33.372379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:33.372379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:33.372379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:33.372379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:33.372379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:33.372379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:33.372379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:33.372379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:33.372379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:33.372379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:33.372379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:41:33.372379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:41:33.372379 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:41:33.373863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:41:33.373863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:33.373863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:33.373863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:33.373863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:33.373863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:33.373863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:33.373863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:33.373863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:33.373863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:33.373863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:33.373863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:33.373863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:33.373863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:33.373863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:33.373863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:33.373863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:33.374751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:41:33.374751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:41:33.374751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:33.374751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:33.381193 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 463/923] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:33.383178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86baa892-2de1-4fe6-96da-370b507a7095 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:33.539571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:33.558720 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86baa892-2de1-4fe6-96da-370b507a7095 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:33.629361 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f6eb04-5d02-410a-a313-087661422320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:33.630806 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6eb04-5d02-410a-a313-087661422320 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:33.631306 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6eb04-5d02-410a-a313-087661422320 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:33.631804 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6eb04-5d02-410a-a313-087661422320 None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:33.633300 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f6eb04-5d02-410a-a313-087661422320 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:33.633910 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f6eb04-5d02-410a-a313-087661422320 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:33.636477 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f6eb04-5d02-410a-a313-087661422320 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:33.662926 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 461/924] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:33.668504 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff33b5a-c192-4d95-a06f-28322319db71 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:33.723283 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:33.805894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff33b5a-c192-4d95-a06f-28322319db71 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:33.868924 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847b611-6e9f-4894-b84f-25d2cda2f66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:33.870295 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847b611-6e9f-4894-b84f-25d2cda2f66d None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:33.870784 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847b611-6e9f-4894-b84f-25d2cda2f66d None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:33.871273 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847b611-6e9f-4894-b84f-25d2cda2f66d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:33.886809 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3847b611-6e9f-4894-b84f-25d2cda2f66d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:33.890181 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3847b611-6e9f-4894-b84f-25d2cda2f66d None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:33.919563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-3847b611-6e9f-4894-b84f-25d2cda2f66d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 17 16:41:33.919563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:33.919563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:33.919563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:33.919563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:33.919563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:33.919563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:33.919563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:33.919563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:33.919563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:33.919563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:33.919563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:33.919563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:41:33.919563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:41:33.919563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:41:33.919563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:41:33.920985 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:33.920985 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:33.920985 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:41:33.920985 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:41:33.920985 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:41:33.920985 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:41:33.920985 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:33.920985 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:33.920985 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:33.920985 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:33.920985 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:33.920985 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:33.920985 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:33.920985 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:33.920985 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:33.920985 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:33.920985 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:33.920985 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:33.921909 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:33.921909 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:33.921909 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:41:33.921909 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:41:33.921909 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:41:33.921909 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:41:33.921909 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:41:33.921909 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:41:33.921909 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 17 16:41:33.921909 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:33.922432 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 464/925] 10.222.0.40 () {56 vars in 1117 bytes} [Sun Jul 17 16:41:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:33.928907 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f6eb04-5d02-410a-a313-087661422320 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:34.049112 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad0ed9-bc7a-47e4-af23-eb1be38f0b14 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:34.049488 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad0ed9-bc7a-47e4-af23-eb1be38f0b14 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:34.049685 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad0ed9-bc7a-47e4-af23-eb1be38f0b14 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:34.052940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 465/926] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:41:34.058150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66917685-4c07-422f-bad1-6c5c999030c0 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:34.058441 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66917685-4c07-422f-bad1-6c5c999030c0 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:34.058737 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66917685-4c07-422f-bad1-6c5c999030c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:34.079282 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-66917685-4c07-422f-bad1-6c5c999030c0 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:34.080501 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-66917685-4c07-422f-bad1-6c5c999030c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:34.120466 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f6eb04-5d02-410a-a313-087661422320 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:34.173295 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 466/927] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 17 16:41:34.180805 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d6c0f-97ea-4bf0-b796-5dfe4f8c6161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:34.180805 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d6c0f-97ea-4bf0-b796-5dfe4f8c6161 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:34.180805 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d6c0f-97ea-4bf0-b796-5dfe4f8c6161 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:34.180805 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3d6c0f-97ea-4bf0-b796-5dfe4f8c6161 None admin] PATH_INFO: `/v3/users` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:34.181162 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3d6c0f-97ea-4bf0-b796-5dfe4f8c6161 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:34.181162 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3d6c0f-97ea-4bf0-b796-5dfe4f8c6161 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:34.184809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3d6c0f-97ea-4bf0-b796-5dfe4f8c6161 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:34.207199 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66917685-4c07-422f-bad1-6c5c999030c0 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:34.217468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 462/928] 10.222.0.40 () {56 vars in 1149 bytes} [Sun Jul 17 16:41:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:34.224738 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d6c0f-97ea-4bf0-b796-5dfe4f8c6161 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:34.237318 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:34.264190 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:34.342694 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3d6c0f-97ea-4bf0-b796-5dfe4f8c6161 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:34.357165 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66917685-4c07-422f-bad1-6c5c999030c0 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:34.382722 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42798300-371a-421f-aa30-43310e452d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:34.383890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42798300-371a-421f-aa30-43310e452d20 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:34.384377 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42798300-371a-421f-aa30-43310e452d20 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:34.385388 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42798300-371a-421f-aa30-43310e452d20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:34.401362 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42798300-371a-421f-aa30-43310e452d20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:34.403166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42798300-371a-421f-aa30-43310e452d20 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:34.408070 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40038dee-98c8-4bf5-8451-8f7683809347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:34.409439 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40038dee-98c8-4bf5-8451-8f7683809347 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:34.409934 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40038dee-98c8-4bf5-8451-8f7683809347 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:34.410436 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40038dee-98c8-4bf5-8451-8f7683809347 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:34.411697 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40038dee-98c8-4bf5-8451-8f7683809347 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:34.414281 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40038dee-98c8-4bf5-8451-8f7683809347 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:34.426347 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-42798300-371a-421f-aa30-43310e452d20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:34.426347 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:34.426347 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:34.426347 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:34.426347 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:34.426347 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:34.426347 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:34.426347 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:34.426347 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:34.426347 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:34.426347 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:34.426347 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:34.426347 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:41:34.426347 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:41:34.426347 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:41:34.427267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:41:34.427267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:34.427267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:34.427267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:34.427267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:34.427267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:34.427267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:34.427267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:34.427267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:34.427267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:34.427267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:34.427267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:34.427267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:34.427267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:34.427267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:34.427267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:34.427267 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:34.428137 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:41:34.428137 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:41:34.428137 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:34.428137 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:34.430163 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 463/929] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:41:34.431774 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5fe83-447b-4a66-bd06-efec52ef82ed None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:34.432279 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5fe83-447b-4a66-bd06-efec52ef82ed None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:34.432766 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5fe83-447b-4a66-bd06-efec52ef82ed None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:34.434361 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 464/930] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:34.435403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-40038dee-98c8-4bf5-8451-8f7683809347 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 17 16:41:34.435403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:34.435403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 17 16:41:34.435403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 17 16:41:34.435403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 17 16:41:34.435403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 17 16:41:34.435403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 17 16:41:34.435403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:34.435403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 17 16:41:34.435403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:34.435403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 17 16:41:34.435403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:34.435403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:34.435403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:34.435403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:34.435403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:34.435403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:34.436365 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:34.436365 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:34.436365 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:34.436365 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:34.436365 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:34.436365 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:34.436365 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 17 16:41:34.436365 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 17 16:41:34.436365 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 17 16:41:34.436365 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 17 16:41:34.436365 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:34.436365 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:34.436365 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:34.436365 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:34.436365 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:34.436365 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:34.438023 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:34.438023 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:34.438023 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:34.438023 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:34.438023 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:34.438023 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:34.438023 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:34.438023 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:34.438023 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:34.438023 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:34.438023 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 17 16:41:34.438023 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:34.438023 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 17 16:41:34.438023 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:34.439983 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5fe83-447b-4a66-bd06-efec52ef82ed None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:34.441679 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 467/931] 10.222.0.40 () {56 vars in 1126 bytes} [Sun Jul 17 16:41:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:41:34.442869 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40038dee-98c8-4bf5-8451-8f7683809347 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:34.520701 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:34.575621 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5fe83-447b-4a66-bd06-efec52ef82ed None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:34.578073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:34.584715 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40038dee-98c8-4bf5-8451-8f7683809347 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:34.622164 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fcc2ea-a590-4baa-b1f3-d3de85f69e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:34.623374 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fcc2ea-a590-4baa-b1f3-d3de85f69e48 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:34.623668 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fcc2ea-a590-4baa-b1f3-d3de85f69e48 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:34.623947 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fcc2ea-a590-4baa-b1f3-d3de85f69e48 None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:34.624816 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fcc2ea-a590-4baa-b1f3-d3de85f69e48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:34.625150 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fcc2ea-a590-4baa-b1f3-d3de85f69e48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:34.626555 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fcc2ea-a590-4baa-b1f3-d3de85f69e48 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:34.637909 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47acd480-3fde-4a23-b137-a3136497eaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:34.639293 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47acd480-3fde-4a23-b137-a3136497eaa2 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:34.639715 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47acd480-3fde-4a23-b137-a3136497eaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:34.640135 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47acd480-3fde-4a23-b137-a3136497eaa2 None admin] PATH_INFO: `/v3/services` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:34.641205 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47acd480-3fde-4a23-b137-a3136497eaa2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:34.641639 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47acd480-3fde-4a23-b137-a3136497eaa2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:34.643421 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47acd480-3fde-4a23-b137-a3136497eaa2 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:34.643615 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 465/932] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:41:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:34.645004 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b44e6a-583b-4d02-8b5a-6c6d7eb4cd0e None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:34.645353 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b44e6a-583b-4d02-8b5a-6c6d7eb4cd0e None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:34.645852 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b44e6a-583b-4d02-8b5a-6c6d7eb4cd0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:34.658929 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 468/933] 10.222.0.40 () {56 vars in 1122 bytes} [Sun Jul 17 16:41:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:34.658929 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47acd480-3fde-4a23-b137-a3136497eaa2 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:34.679497 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-f2b44e6a-583b-4d02-8b5a-6c6d7eb4cd0e None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:34.680619 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-f2b44e6a-583b-4d02-8b5a-6c6d7eb4cd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:34.761426 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 466/934] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:41:34.769167 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b44e6a-583b-4d02-8b5a-6c6d7eb4cd0e None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:34.833422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47acd480-3fde-4a23-b137-a3136497eaa2 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:34.866493 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:34.876241 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c8710-11fc-43fc-afb2-9bd87fc7e1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:34.883857 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c8710-11fc-43fc-afb2-9bd87fc7e1c1 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:34.883857 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c8710-11fc-43fc-afb2-9bd87fc7e1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:34.883857 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c8710-11fc-43fc-afb2-9bd87fc7e1c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:34.886275 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c8710-11fc-43fc-afb2-9bd87fc7e1c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:34.887748 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c8710-11fc-43fc-afb2-9bd87fc7e1c1 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:34.903394 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-642c8710-11fc-43fc-afb2-9bd87fc7e1c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:41:34.903394 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:34.903394 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:34.903394 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:34.903394 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:34.903394 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:34.903394 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:34.903394 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:34.903394 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:34.903394 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:34.903394 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:34.903394 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:34.903394 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:41:34.903394 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:41:34.903394 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:41:34.905183 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:41:34.905183 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:34.905183 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:34.905183 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:34.905183 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:34.905183 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:34.905183 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:34.905183 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:34.905183 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:34.905183 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:34.905183 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:34.905183 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:34.905183 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:34.905183 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:34.905183 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:34.905183 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:34.905183 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:34.906505 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:41:34.906505 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:41:34.906505 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:41:34.906505 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:41:34.906505 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:41:34.906505 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:41:34.906505 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:41:34.906505 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:34.906505 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 469/935] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:41:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:34.906505 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c8710-11fc-43fc-afb2-9bd87fc7e1c1 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:34.935594 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:34.940814 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b44e6a-583b-4d02-8b5a-6c6d7eb4cd0e None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:34.983669 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b3b82-d1b7-4da2-95e7-4d0e5bef4f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:34.984349 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b3b82-d1b7-4da2-95e7-4d0e5bef4f93 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:34.984640 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b3b82-d1b7-4da2-95e7-4d0e5bef4f93 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:34.984955 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b3b82-d1b7-4da2-95e7-4d0e5bef4f93 None admin] PATH_INFO: `/v3/services` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:34.985682 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921b3b82-d1b7-4da2-95e7-4d0e5bef4f93 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:34.986229 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921b3b82-d1b7-4da2-95e7-4d0e5bef4f93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:34.987380 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921b3b82-d1b7-4da2-95e7-4d0e5bef4f93 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:34.992758 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 467/936] 10.222.0.40 () {56 vars in 1122 bytes} [Sun Jul 17 16:41:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:34.993622 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b3b82-d1b7-4da2-95e7-4d0e5bef4f93 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:35.024586 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c8710-11fc-43fc-afb2-9bd87fc7e1c1 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:35.069496 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbde8fc-3564-400a-9f9f-c6db3a9e0150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:35.070952 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbde8fc-3564-400a-9f9f-c6db3a9e0150 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:35.071478 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbde8fc-3564-400a-9f9f-c6db3a9e0150 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:35.071940 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbde8fc-3564-400a-9f9f-c6db3a9e0150 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:35.073202 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbde8fc-3564-400a-9f9f-c6db3a9e0150 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:35.076222 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbde8fc-3564-400a-9f9f-c6db3a9e0150 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:35.101998 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-8cbde8fc-3564-400a-9f9f-c6db3a9e0150 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 17 16:41:35.101998 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:35.101998 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 17 16:41:35.101998 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 17 16:41:35.101998 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 17 16:41:35.101998 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 17 16:41:35.101998 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 17 16:41:35.101998 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:35.101998 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 17 16:41:35.101998 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:35.101998 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 17 16:41:35.101998 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:35.101998 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:35.101998 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:35.101998 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:35.101998 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:35.103062 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:35.103062 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:35.103062 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:35.103062 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:35.103062 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:35.103062 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:35.103062 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:35.103062 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 17 16:41:35.103062 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 17 16:41:35.103062 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 17 16:41:35.103062 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 17 16:41:35.103062 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:35.103062 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:35.103062 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:35.103062 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:35.103062 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:35.103906 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:35.103906 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:35.103906 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:35.103906 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:35.103906 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:35.103906 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:35.103906 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:35.103906 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:35.103906 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:35.103906 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:35.103906 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:35.103906 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 17 16:41:35.103906 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:35.103906 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 17 16:41:35.103906 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:35.104687 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 470/937] 10.222.0.40 () {56 vars in 1132 bytes} [Sun Jul 17 16:41:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:35.107414 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a78cc-2ffa-4d9f-a6ac-9b46f66b5989 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:35.107946 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a78cc-2ffa-4d9f-a6ac-9b46f66b5989 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:35.108350 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a78cc-2ffa-4d9f-a6ac-9b46f66b5989 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:35.112985 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b3b82-d1b7-4da2-95e7-4d0e5bef4f93 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:35.113170 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 471/938] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:35.113170 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a78cc-2ffa-4d9f-a6ac-9b46f66b5989 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:35.127153 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:35.148460 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b0ed7-1435-4d74-b0b5-8abe949107a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:35.149163 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b0ed7-1435-4d74-b0b5-8abe949107a1 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:35.149614 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b0ed7-1435-4d74-b0b5-8abe949107a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:35.149906 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7b0ed7-1435-4d74-b0b5-8abe949107a1 None admin] PATH_INFO: `/v3/projects` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:35.150715 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b0ed7-1435-4d74-b0b5-8abe949107a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:35.151029 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b0ed7-1435-4d74-b0b5-8abe949107a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:35.152480 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7b0ed7-1435-4d74-b0b5-8abe949107a1 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:35.166971 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 468/939] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:41:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:35.168277 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b0ed7-1435-4d74-b0b5-8abe949107a1 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:35.236776 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:35.254972 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a78cc-2ffa-4d9f-a6ac-9b46f66b5989 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:35.292978 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c680838a-e195-414a-8b14-92e7ca75a0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:35.293638 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c680838a-e195-414a-8b14-92e7ca75a0a9 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:35.293933 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c680838a-e195-414a-8b14-92e7ca75a0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:35.294252 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c680838a-e195-414a-8b14-92e7ca75a0a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:35.295685 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c680838a-e195-414a-8b14-92e7ca75a0a9 None admin] RBAC: Adding query filter params (interface=public, service_id=2151f51760474456a8080af98b21070a, region_id=RegionOne) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:35.296006 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c680838a-e195-414a-8b14-92e7ca75a0a9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:35.296339 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7b0ed7-1435-4d74-b0b5-8abe949107a1 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:35.297450 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c680838a-e195-414a-8b14-92e7ca75a0a9 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:35.307541 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 472/940] 10.222.0.40 () {56 vars in 1251 bytes} [Sun Jul 17 16:41:35 2022] GET /identity/v3/endpoints?service_id=2151f51760474456a8080af98b21070a&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:35.307541 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3fbb2f-7600-40b0-82ba-352e11c1f582 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:35.307541 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3fbb2f-7600-40b0-82ba-352e11c1f582 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:35.307541 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3fbb2f-7600-40b0-82ba-352e11c1f582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:35.323441 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-7c3fbb2f-7600-40b0-82ba-352e11c1f582 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:35.324665 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-7c3fbb2f-7600-40b0-82ba-352e11c1f582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:35.335353 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72f095-54c8-4cc6-84e2-b70ed87f7675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:35.336612 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72f095-54c8-4cc6-84e2-b70ed87f7675 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:35.337418 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72f095-54c8-4cc6-84e2-b70ed87f7675 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:35.337713 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72f095-54c8-4cc6-84e2-b70ed87f7675 None admin] PATH_INFO: `/v3/services` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:35.338448 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc72f095-54c8-4cc6-84e2-b70ed87f7675 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:35.338867 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc72f095-54c8-4cc6-84e2-b70ed87f7675 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:35.340501 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc72f095-54c8-4cc6-84e2-b70ed87f7675 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:35.346326 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 469/941] 10.222.0.40 () {56 vars in 1126 bytes} [Sun Jul 17 16:41:35 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:35.347562 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72f095-54c8-4cc6-84e2-b70ed87f7675 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:35.394924 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:35.404888 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 473/942] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:41:35.405876 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3fbb2f-7600-40b0-82ba-352e11c1f582 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:35.484070 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72f095-54c8-4cc6-84e2-b70ed87f7675 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:35.507885 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:35.526997 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3fbb2f-7600-40b0-82ba-352e11c1f582 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:35.536302 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfffd0e-7ff2-40a4-a72f-97ca8f2abe1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:35.536302 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfffd0e-7ff2-40a4-a72f-97ca8f2abe1a None admin] REQUEST_METHOD: `PUT` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:35.536302 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfffd0e-7ff2-40a4-a72f-97ca8f2abe1a None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:35.536302 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfffd0e-7ff2-40a4-a72f-97ca8f2abe1a None admin] PATH_INFO: `/v3/projects/927057f9a5af4833aeac9992203a223c/users/37ed5f72dc8c4a5685bd1245cadcdf7e/roles/68d3a95a9fc34fd1a28a9861d22ee828` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:35.536658 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfffd0e-7ff2-40a4-a72f-97ca8f2abe1a None admin] RBAC: Authorizing `identity:create_grant(project_id=927057f9a5af4833aeac9992203a223c, user_id=37ed5f72dc8c4a5685bd1245cadcdf7e, role_id=68d3a95a9fc34fd1a28a9861d22ee828)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:35.536658 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfffd0e-7ff2-40a4-a72f-97ca8f2abe1a None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:35.565414 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69493c5c-78af-44a0-b948-8e776a24b278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:35.566077 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69493c5c-78af-44a0-b948-8e776a24b278 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:35.566366 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69493c5c-78af-44a0-b948-8e776a24b278 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:35.566643 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69493c5c-78af-44a0-b948-8e776a24b278 None admin] PATH_INFO: `/v3/services` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:35.567593 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69493c5c-78af-44a0-b948-8e776a24b278 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:35.569007 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69493c5c-78af-44a0-b948-8e776a24b278 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:35.575436 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 474/943] 10.222.0.40 () {56 vars in 1087 bytes} [Sun Jul 17 16:41:35 2022] GET /identity/v3/services => generated 1497 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 17 16:41:35.577007 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69493c5c-78af-44a0-b948-8e776a24b278 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:35.637511 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 470/944] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 17 16:41:35 2022] PUT /identity/v3/projects/927057f9a5af4833aeac9992203a223c/users/37ed5f72dc8c4a5685bd1245cadcdf7e/roles/68d3a95a9fc34fd1a28a9861d22ee828 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 17 16:41:35.639229 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfffd0e-7ff2-40a4-a72f-97ca8f2abe1a None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:35.735492 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69493c5c-78af-44a0-b948-8e776a24b278 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:35.789355 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4d8c0-dbb7-4b1f-897b-baa98fc54f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:35.790986 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4d8c0-dbb7-4b1f-897b-baa98fc54f76 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:35.791817 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4d8c0-dbb7-4b1f-897b-baa98fc54f76 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:35.792599 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4d8c0-dbb7-4b1f-897b-baa98fc54f76 None admin] PATH_INFO: `/v3/services` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:35.794072 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb4d8c0-dbb7-4b1f-897b-baa98fc54f76 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:35.795014 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb4d8c0-dbb7-4b1f-897b-baa98fc54f76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:35.798066 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb4d8c0-dbb7-4b1f-897b-baa98fc54f76 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:35.807337 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 475/945] 10.222.0.40 () {56 vars in 1126 bytes} [Sun Jul 17 16:41:35 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:35.819184 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4d8c0-dbb7-4b1f-897b-baa98fc54f76 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:35.834381 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfffd0e-7ff2-40a4-a72f-97ca8f2abe1a None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:35.836671 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:35.844269 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:35.885968 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b78d8-f422-4e35-a0ca-909b51e658cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:35.889728 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b78d8-f422-4e35-a0ca-909b51e658cb None admin] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:35.890012 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b78d8-f422-4e35-a0ca-909b51e658cb None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:35.890296 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b78d8-f422-4e35-a0ca-909b51e658cb None admin] PATH_INFO: `/v3/projects` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:35.891840 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7b78d8-f422-4e35-a0ca-909b51e658cb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:35.893450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7b78d8-f422-4e35-a0ca-909b51e658cb None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:35.894886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: WARNING keystone.server.flask.common [None req-3a7b78d8-f422-4e35-a0ca-909b51e658cb None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 17 16:41:35.959205 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.sql.core [None req-3a7b78d8-f422-4e35-a0ca-909b51e658cb None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 17 16:41:35.959205 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 17 16:41:35.959205 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [parameters: {'id': '6fd50cee4b0349368ff6d6e53b53a5c1', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 17 16:41:35.959205 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87199) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 17 16:41:35.972433 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-3a7b78d8-f422-4e35-a0ca-909b51e658cb None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6fd50cee4b0349368ff6d6e53b53a5c1.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6fd50cee4b0349368ff6d6e53b53a5c1. Jul 17 16:41:35.972433 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:35.972433 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 17 16:41:35.972433 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 17 16:41:35.972433 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 17 16:41:35.972433 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 17 16:41:35.972433 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 17 16:41:35.972433 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application result = self._query(query) Jul 17 16:41:35.972433 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 17 16:41:35.972433 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application conn.query(q) Jul 17 16:41:35.972433 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 17 16:41:35.972433 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 17 16:41:35.972433 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 17 16:41:35.974100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application result.read() Jul 17 16:41:35.974100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 17 16:41:35.974100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 17 16:41:35.974100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 17 16:41:35.974100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 17 16:41:35.974100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 17 16:41:35.974100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 17 16:41:35.974100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 17 16:41:35.974100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 17 16:41:35.974100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 17 16:41:35.974100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:35.974100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 17 16:41:35.974100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:35.974100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:35.974100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 17 16:41:35.974100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 17 16:41:35.974100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 17 16:41:35.974100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 17 16:41:35.977713 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 17 16:41:35.977713 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application next(self.gen) Jul 17 16:41:35.977713 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 17 16:41:35.977713 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application yield resource Jul 17 16:41:35.977713 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 17 16:41:35.977713 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application next(self.gen) Jul 17 16:41:35.977713 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 17 16:41:35.977713 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self.session.rollback() Jul 17 16:41:35.977713 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 17 16:41:35.977713 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self.force_reraise() Jul 17 16:41:35.977713 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 17 16:41:35.977713 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise self.value Jul 17 16:41:35.977713 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 17 16:41:35.977713 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 17 16:41:35.977713 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 17 16:41:35.977713 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application session.commit() Jul 17 16:41:35.977713 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 17 16:41:35.981280 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 17 16:41:35.981280 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 17 16:41:35.981280 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self._prepare_impl() Jul 17 16:41:35.981280 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 17 16:41:35.981280 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self.session.flush() Jul 17 16:41:35.981280 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 17 16:41:35.981280 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self._flush(objects) Jul 17 16:41:35.981280 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 17 16:41:35.981280 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 17 16:41:35.981280 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 17 16:41:35.981280 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application compat.raise_( Jul 17 16:41:35.981280 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 17 16:41:35.981280 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception Jul 17 16:41:35.981280 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 17 16:41:35.981280 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application flush_context.execute() Jul 17 16:41:35.981280 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 17 16:41:35.981280 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rec.execute(self) Jul 17 16:41:35.985205 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 17 16:41:35.985205 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 17 16:41:35.985205 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 17 16:41:35.985205 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 17 16:41:35.985205 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 17 16:41:35.985205 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 17 16:41:35.985205 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 17 16:41:35.985205 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 17 16:41:35.985205 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 17 16:41:35.985205 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 17 16:41:35.985205 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 17 16:41:35.985205 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 17 16:41:35.985205 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 17 16:41:35.985205 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 17 16:41:35.985205 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 17 16:41:35.988088 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 17 16:41:35.988088 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 17 16:41:35.988088 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception Jul 17 16:41:35.988088 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 17 16:41:35.988088 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 17 16:41:35.988088 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 17 16:41:35.988088 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 17 16:41:35.988088 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 17 16:41:35.988088 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application result = self._query(query) Jul 17 16:41:35.988088 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 17 16:41:35.988088 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application conn.query(q) Jul 17 16:41:35.988088 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 17 16:41:35.988088 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 17 16:41:35.988088 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 17 16:41:35.988088 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application result.read() Jul 17 16:41:35.988088 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 17 16:41:35.988088 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 17 16:41:35.991737 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 17 16:41:35.991737 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 17 16:41:35.991737 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 17 16:41:35.991737 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 17 16:41:35.991737 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 17 16:41:35.991737 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 17 16:41:35.991737 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 17 16:41:35.991737 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 17 16:41:35.991737 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [parameters: {'id': '6fd50cee4b0349368ff6d6e53b53a5c1', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 17 16:41:35.991737 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 17 16:41:35.991737 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:35.991737 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 17 16:41:35.991737 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:35.991737 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:35.991737 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 17 16:41:35.994450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 17 16:41:35.994450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 17 16:41:35.994450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 17 16:41:35.994450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 17 16:41:35.994450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:35.994450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 17 16:41:35.994450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:35.994450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:35.994450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:35.994450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:35.994450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:35.994450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:35.994450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:35.994450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:35.994450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:35.994450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:35.994450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:35.996774 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:35.996774 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 17 16:41:35.996774 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 17 16:41:35.996774 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:35.996774 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:35.996774 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 17 16:41:35.996774 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 17 16:41:35.996774 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6fd50cee4b0349368ff6d6e53b53a5c1. Jul 17 16:41:35.996774 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:35.996774 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 471/946] 10.222.0.40 () {60 vars in 1140 bytes} [Sun Jul 17 16:41:35 2022] POST /identity/v3/projects => generated 262 bytes in 337 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:35.996774 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b78d8-f422-4e35-a0ca-909b51e658cb None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:36.021978 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4d8c0-dbb7-4b1f-897b-baa98fc54f76 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:36.073297 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f02f49-de1e-468b-be55-c9404e8b0899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:36.074077 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f02f49-de1e-468b-be55-c9404e8b0899 None admin] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:36.075049 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f02f49-de1e-468b-be55-c9404e8b0899 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:36.075049 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f02f49-de1e-468b-be55-c9404e8b0899 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:36.076381 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f02f49-de1e-468b-be55-c9404e8b0899 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:36.077989 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f02f49-de1e-468b-be55-c9404e8b0899 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:36.163512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 476/947] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 17 16:41:35 2022] POST /identity/v3/endpoints => generated 347 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:41:36.197105 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b78d8-f422-4e35-a0ca-909b51e658cb None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:36.290800 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3645ca-f4d2-42de-86b9-c1f0ec13620b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:36.292151 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3645ca-f4d2-42de-86b9-c1f0ec13620b None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:36.292629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3645ca-f4d2-42de-86b9-c1f0ec13620b None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:36.293138 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3645ca-f4d2-42de-86b9-c1f0ec13620b None admin] PATH_INFO: `/v3/projects/service` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:36.307765 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3645ca-f4d2-42de-86b9-c1f0ec13620b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:36.311331 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3645ca-f4d2-42de-86b9-c1f0ec13620b None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:36.340120 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-bf3645ca-f4d2-42de-86b9-c1f0ec13620b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:41:36.340120 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:36.340120 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:36.340120 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:36.340120 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:36.340120 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:36.340120 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:36.340120 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:36.340120 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:36.340120 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:36.340120 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:36.340120 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:36.340120 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:41:36.340120 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:41:36.340120 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:41:36.341220 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:41:36.341220 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:36.341220 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:36.341220 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:36.341220 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:36.341220 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:36.341220 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:36.341220 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:36.341220 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:36.341220 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:36.341220 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:36.341220 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:36.341220 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:36.341220 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:36.341220 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:36.341220 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:36.341220 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:36.342092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:41:36.342092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:41:36.342092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:41:36.342092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:41:36.342092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:41:36.342092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:41:36.342092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:41:36.342092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:36.342092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 472/948] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:41:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:36.349824 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f02f49-de1e-468b-be55-c9404e8b0899 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:36.438793 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:36.481608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f02f49-de1e-468b-be55-c9404e8b0899 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:36.547245 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cf8f32-c8e9-4eda-8252-23ad7dc562ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:36.548752 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf8f32-c8e9-4eda-8252-23ad7dc562ed None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:36.549518 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf8f32-c8e9-4eda-8252-23ad7dc562ed None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:36.550238 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cf8f32-c8e9-4eda-8252-23ad7dc562ed None admin] PATH_INFO: `/v3/projects` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:36.551914 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cf8f32-c8e9-4eda-8252-23ad7dc562ed None admin] RBAC: Adding query filter params (name=service) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:36.552694 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cf8f32-c8e9-4eda-8252-23ad7dc562ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:36.555704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cf8f32-c8e9-4eda-8252-23ad7dc562ed None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:36.593460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 477/949] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Jul 17 16:41:36 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:38.019071 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b3d99-f4fa-447c-9fd8-be0ebf942b51 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:38.019750 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b3d99-f4fa-447c-9fd8-be0ebf942b51 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:38.019750 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b3d99-f4fa-447c-9fd8-be0ebf942b51 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:38.021352 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 473/950] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:38.027015 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e024a2-2544-455f-b304-773f2b93d622 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:38.027505 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e024a2-2544-455f-b304-773f2b93d622 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:38.027609 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e024a2-2544-455f-b304-773f2b93d622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:38.046786 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-74e024a2-2544-455f-b304-773f2b93d622 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:38.048000 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-74e024a2-2544-455f-b304-773f2b93d622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:38.126501 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 478/951] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:41:38.150216 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b3d99-f4fa-447c-9fd8-be0ebf942b51 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:38.164145 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:38.269232 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b3d99-f4fa-447c-9fd8-be0ebf942b51 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:38.307738 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0825fa18-45d5-4651-8050-51062be4169d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:38.308567 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825fa18-45d5-4651-8050-51062be4169d None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:38.308875 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825fa18-45d5-4651-8050-51062be4169d None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:38.309155 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0825fa18-45d5-4651-8050-51062be4169d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:38.322223 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0825fa18-45d5-4651-8050-51062be4169d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:41:38.331678 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-0825fa18-45d5-4651-8050-51062be4169d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 17 16:41:38.331678 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:38.331678 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:38.331678 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:38.331678 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:38.331678 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:38.331678 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:38.331678 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:38.331678 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:38.331678 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:38.331678 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:38.331678 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:38.331678 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:41:38.331678 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:41:38.331678 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:41:38.331678 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:41:38.332764 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:41:38.332764 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:41:38.332764 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:41:38.332764 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:41:38.332764 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:38.332764 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:38.332764 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:38.332764 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:38.332764 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:38.332764 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:38.332764 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:38.332764 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:38.332764 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:38.332764 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:38.332764 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:38.332764 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:38.333878 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:38.333878 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:38.333878 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:38.333878 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:38.333878 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:41:38.333878 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:41:38.333878 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:41:38.333878 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:41:38.333878 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:41:38.333878 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:41:38.333878 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 17 16:41:38.333878 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:38.333878 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 474/952] 10.222.0.40 () {56 vars in 1120 bytes} [Sun Jul 17 16:41:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:38.339286 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e024a2-2544-455f-b304-773f2b93d622 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:38.404360 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:38.433686 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966bdd7-4573-4bea-a7cd-44ac0c3d73d7 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:38.433996 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966bdd7-4573-4bea-a7cd-44ac0c3d73d7 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:38.434249 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6966bdd7-4573-4bea-a7cd-44ac0c3d73d7 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:38.436921 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 475/953] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:38.443075 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c239177-6de7-4f21-b829-88239768bf13 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:38.443325 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c239177-6de7-4f21-b829-88239768bf13 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:38.443600 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c239177-6de7-4f21-b829-88239768bf13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:38.461904 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e024a2-2544-455f-b304-773f2b93d622 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:38.462581 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-9c239177-6de7-4f21-b829-88239768bf13 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:38.464156 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-9c239177-6de7-4f21-b829-88239768bf13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:38.506670 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c848b-f03d-42a9-ae88-48f6ab2f2aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:38.507792 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c848b-f03d-42a9-ae88-48f6ab2f2aed None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:38.508281 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c848b-f03d-42a9-ae88-48f6ab2f2aed None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:38.508768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c848b-f03d-42a9-ae88-48f6ab2f2aed None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:38.509806 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c848b-f03d-42a9-ae88-48f6ab2f2aed None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:38.510320 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c848b-f03d-42a9-ae88-48f6ab2f2aed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:38.511960 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c848b-f03d-42a9-ae88-48f6ab2f2aed None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:38.523929 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 479/954] 10.222.0.40 () {56 vars in 1115 bytes} [Sun Jul 17 16:41:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:38.530032 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c848b-f03d-42a9-ae88-48f6ab2f2aed None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:38.548669 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:38.560950 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 476/955] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 17 16:41:38.580182 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c239177-6de7-4f21-b829-88239768bf13 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:38.672706 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c848b-f03d-42a9-ae88-48f6ab2f2aed None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:38.694964 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:38.710059 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c239177-6de7-4f21-b829-88239768bf13 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:38.718883 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73338ea8-df63-48df-95a0-69fccd30580c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:38.719795 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73338ea8-df63-48df-95a0-69fccd30580c None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:38.720187 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73338ea8-df63-48df-95a0-69fccd30580c None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:38.720526 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73338ea8-df63-48df-95a0-69fccd30580c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:38.733707 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73338ea8-df63-48df-95a0-69fccd30580c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:38.735218 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73338ea8-df63-48df-95a0-69fccd30580c None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:38.760620 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdbdaf-a385-4fc2-8d75-cb50e24599a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:38.761589 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdbdaf-a385-4fc2-8d75-cb50e24599a7 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:38.761869 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdbdaf-a385-4fc2-8d75-cb50e24599a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:38.762147 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdbdaf-a385-4fc2-8d75-cb50e24599a7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:38.763416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbdbdaf-a385-4fc2-8d75-cb50e24599a7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:38.763956 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-73338ea8-df63-48df-95a0-69fccd30580c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:38.763956 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:38.763956 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:38.763956 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:38.763956 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:38.763956 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:38.763956 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:38.763956 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:38.763956 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:38.763956 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:38.763956 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:38.763956 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:38.763956 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:41:38.763956 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:41:38.763956 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:41:38.765147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:41:38.765147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:38.765147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:38.765147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:38.765147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:38.765147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:38.765147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:38.765147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:38.765147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:38.765147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:38.765147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:38.765147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:38.765147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:38.765147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:38.765147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:38.765147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:38.765147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:38.766103 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:41:38.766103 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:41:38.766103 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:38.766103 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:38.766339 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbdbdaf-a385-4fc2-8d75-cb50e24599a7 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:38.766967 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 480/956] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:38.769342 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5536d49c-1157-48ec-ab88-5dc3080a7c1f None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:38.769840 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5536d49c-1157-48ec-ab88-5dc3080a7c1f None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:38.770336 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5536d49c-1157-48ec-ab88-5dc3080a7c1f None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:38.777388 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 481/957] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:38.777388 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5536d49c-1157-48ec-ab88-5dc3080a7c1f None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:38.789273 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-5fbdbdaf-a385-4fc2-8d75-cb50e24599a7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 17 16:41:38.789273 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:38.789273 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 17 16:41:38.789273 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 17 16:41:38.789273 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 17 16:41:38.789273 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 17 16:41:38.789273 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 17 16:41:38.789273 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:38.789273 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 17 16:41:38.789273 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:38.789273 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 17 16:41:38.789273 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:38.789273 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:38.789273 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:38.789273 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:38.789273 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:38.789273 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:38.792366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:38.792366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:38.792366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:38.792366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:38.792366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:38.792366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:38.792366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 17 16:41:38.792366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 17 16:41:38.792366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 17 16:41:38.792366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 17 16:41:38.792366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:38.792366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:38.792366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:38.792366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:38.792366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:38.792366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:38.796148 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:38.796148 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:38.796148 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:38.796148 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:38.796148 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:38.796148 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:38.796148 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:38.796148 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:38.796148 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:38.796148 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:38.796148 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 17 16:41:38.796148 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:38.796148 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 17 16:41:38.796148 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:38.799135 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 477/958] 10.222.0.40 () {56 vars in 1126 bytes} [Sun Jul 17 16:41:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:41:38.799135 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8d804-344d-41d9-89e7-182500911edf None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:38.799135 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8d804-344d-41d9-89e7-182500911edf None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:38.799135 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8d804-344d-41d9-89e7-182500911edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:38.818248 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-d0a8d804-344d-41d9-89e7-182500911edf None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:38.819458 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-d0a8d804-344d-41d9-89e7-182500911edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:38.889282 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:38.904874 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 478/959] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:41:38.904874 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8d804-344d-41d9-89e7-182500911edf None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:38.949941 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5536d49c-1157-48ec-ab88-5dc3080a7c1f None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:38.994732 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92a89f-dbcd-4467-921a-f119c2f7766e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:38.995878 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92a89f-dbcd-4467-921a-f119c2f7766e None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:38.996375 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92a89f-dbcd-4467-921a-f119c2f7766e None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:38.996909 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92a89f-dbcd-4467-921a-f119c2f7766e None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:38.998857 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af92a89f-dbcd-4467-921a-f119c2f7766e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:38.999411 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af92a89f-dbcd-4467-921a-f119c2f7766e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:39.002111 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af92a89f-dbcd-4467-921a-f119c2f7766e None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:39.023853 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 482/960] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:41:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:39.025391 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92a89f-dbcd-4467-921a-f119c2f7766e None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:39.062595 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8d804-344d-41d9-89e7-182500911edf None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:39.079495 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:39.119507 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae45004e-769d-4770-8433-68bbd143952f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:39.120635 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45004e-769d-4770-8433-68bbd143952f None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:39.121173 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45004e-769d-4770-8433-68bbd143952f None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:39.121725 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45004e-769d-4770-8433-68bbd143952f None admin] PATH_INFO: `/v3/services` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:39.123495 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae45004e-769d-4770-8433-68bbd143952f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:39.124037 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae45004e-769d-4770-8433-68bbd143952f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:39.126521 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae45004e-769d-4770-8433-68bbd143952f None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:39.136434 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 479/961] 10.222.0.40 () {56 vars in 1122 bytes} [Sun Jul 17 16:41:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:39.137803 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae45004e-769d-4770-8433-68bbd143952f None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:39.223586 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92a89f-dbcd-4467-921a-f119c2f7766e None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:39.229611 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:39.289441 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c63499-06f1-4d48-a856-3b784cdb74e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:39.290748 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae45004e-769d-4770-8433-68bbd143952f None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:39.290859 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c63499-06f1-4d48-a856-3b784cdb74e9 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:39.291312 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c63499-06f1-4d48-a856-3b784cdb74e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:39.291793 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c63499-06f1-4d48-a856-3b784cdb74e9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:39.293061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c63499-06f1-4d48-a856-3b784cdb74e9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:39.296002 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c63499-06f1-4d48-a856-3b784cdb74e9 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:39.319878 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-18c63499-06f1-4d48-a856-3b784cdb74e9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 17 16:41:39.319878 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:39.319878 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 17 16:41:39.319878 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 17 16:41:39.319878 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 17 16:41:39.319878 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 17 16:41:39.319878 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 17 16:41:39.319878 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:39.319878 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 17 16:41:39.319878 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:39.319878 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 17 16:41:39.319878 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:39.319878 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:39.319878 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:39.319878 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:39.319878 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:39.319878 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:39.321252 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:39.321252 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:39.321252 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:39.321252 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:39.321252 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:39.321252 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:39.321252 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 17 16:41:39.321252 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 17 16:41:39.321252 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 17 16:41:39.321252 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 17 16:41:39.321252 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:39.321252 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:39.321252 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:39.321252 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:39.321252 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:39.321252 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:39.322108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:39.322108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:39.322108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:39.322108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:39.322108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:39.322108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:39.322108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:39.322108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:39.322108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:39.322108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:39.322108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 17 16:41:39.322108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:39.322108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 17 16:41:39.322108 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:39.329294 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 483/962] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:41:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:39.332396 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c63499-06f1-4d48-a856-3b784cdb74e9 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:39.344718 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77654a57-0c23-4409-8040-b40504fb48aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:39.346372 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77654a57-0c23-4409-8040-b40504fb48aa None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:39.347100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77654a57-0c23-4409-8040-b40504fb48aa None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:39.347746 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77654a57-0c23-4409-8040-b40504fb48aa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:39.363184 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77654a57-0c23-4409-8040-b40504fb48aa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:39.366075 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77654a57-0c23-4409-8040-b40504fb48aa None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:39.398543 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-77654a57-0c23-4409-8040-b40504fb48aa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 17 16:41:39.398543 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:39.398543 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:39.398543 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:39.398543 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:39.398543 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:39.398543 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:39.398543 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:39.398543 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:39.398543 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:39.398543 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:39.398543 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:39.398543 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:41:39.398543 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:41:39.398543 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:41:39.398543 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:41:39.399549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:39.399549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:39.399549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:41:39.399549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:41:39.399549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:41:39.399549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:41:39.399549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:39.399549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:39.399549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:39.399549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:39.399549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:39.399549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:39.399549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:39.399549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:39.399549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:39.399549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:39.399549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:39.399549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:39.400475 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:39.400475 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:39.400475 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:41:39.400475 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:41:39.400475 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:41:39.400475 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:41:39.400475 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:41:39.400475 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:41:39.400475 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 17 16:41:39.400475 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:39.405254 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 480/963] 10.222.0.40 () {56 vars in 1117 bytes} [Sun Jul 17 16:41:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:39.408005 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77654a57-0c23-4409-8040-b40504fb48aa None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:39.486670 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:39.500820 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c63499-06f1-4d48-a856-3b784cdb74e9 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:39.542550 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b37f28f-685c-49ba-9006-24daab0d2723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:39.543229 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37f28f-685c-49ba-9006-24daab0d2723 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:39.543511 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37f28f-685c-49ba-9006-24daab0d2723 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:39.543785 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37f28f-685c-49ba-9006-24daab0d2723 None admin] PATH_INFO: `/v3/services` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:39.545081 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b37f28f-685c-49ba-9006-24daab0d2723 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:39.545388 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b37f28f-685c-49ba-9006-24daab0d2723 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:39.546796 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b37f28f-685c-49ba-9006-24daab0d2723 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:39.552915 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 484/964] 10.222.0.40 () {56 vars in 1122 bytes} [Sun Jul 17 16:41:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:39.553607 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b37f28f-685c-49ba-9006-24daab0d2723 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:39.591949 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77654a57-0c23-4409-8040-b40504fb48aa None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:39.628347 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:39.647689 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da59261-ec94-4406-9286-e6bbdf028e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:39.648443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da59261-ec94-4406-9286-e6bbdf028e4a None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:39.648839 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da59261-ec94-4406-9286-e6bbdf028e4a None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:39.649235 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da59261-ec94-4406-9286-e6bbdf028e4a None admin] PATH_INFO: `/v3/services` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:39.650050 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da59261-ec94-4406-9286-e6bbdf028e4a None admin] RBAC: Adding query filter params (name=compute) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:39.650450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da59261-ec94-4406-9286-e6bbdf028e4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:39.652112 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da59261-ec94-4406-9286-e6bbdf028e4a None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:39.658676 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 481/965] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Jul 17 16:41:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:39.662313 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da59261-ec94-4406-9286-e6bbdf028e4a None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:39.684117 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b37f28f-685c-49ba-9006-24daab0d2723 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:39.727690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9401c7fd-1728-40a5-a85e-aa916a53518f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:39.728815 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401c7fd-1728-40a5-a85e-aa916a53518f None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:39.729285 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401c7fd-1728-40a5-a85e-aa916a53518f None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:39.729741 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401c7fd-1728-40a5-a85e-aa916a53518f None admin] PATH_INFO: `/v3/users` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:39.731261 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9401c7fd-1728-40a5-a85e-aa916a53518f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:39.731656 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9401c7fd-1728-40a5-a85e-aa916a53518f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:39.734182 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9401c7fd-1728-40a5-a85e-aa916a53518f None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:39.772980 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 485/966] 10.222.0.40 () {56 vars in 1149 bytes} [Sun Jul 17 16:41:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:39.772980 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b613e86-3ed6-4d55-89ed-756537365202 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:39.772980 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b613e86-3ed6-4d55-89ed-756537365202 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:39.772980 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b613e86-3ed6-4d55-89ed-756537365202 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:39.772980 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 486/967] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:39.772980 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b613e86-3ed6-4d55-89ed-756537365202 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:39.795883 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:39.810015 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da59261-ec94-4406-9286-e6bbdf028e4a None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:39.849646 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f13d7-eb59-4e2a-8bb5-5cfd520571e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:39.850229 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f13d7-eb59-4e2a-8bb5-5cfd520571e2 None admin] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:39.850517 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f13d7-eb59-4e2a-8bb5-5cfd520571e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:39.850803 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f13d7-eb59-4e2a-8bb5-5cfd520571e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:39.851552 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2f13d7-eb59-4e2a-8bb5-5cfd520571e2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:39.852987 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2f13d7-eb59-4e2a-8bb5-5cfd520571e2 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:39.929558 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b613e86-3ed6-4d55-89ed-756537365202 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:39.962370 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 482/968] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 17 16:41:39 2022] POST /identity/v3/endpoints => generated 349 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:41:39.964236 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f13d7-eb59-4e2a-8bb5-5cfd520571e2 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:39.995669 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aba474-1362-4dce-9789-11b2401500ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:39.997388 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aba474-1362-4dce-9789-11b2401500ef None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:39.998159 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aba474-1362-4dce-9789-11b2401500ef None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:39.999156 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aba474-1362-4dce-9789-11b2401500ef None admin] PATH_INFO: `/v3/services` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:40.000639 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aba474-1362-4dce-9789-11b2401500ef None admin] RBAC: Adding query filter params (type=compute) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:40.001246 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aba474-1362-4dce-9789-11b2401500ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:40.004032 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aba474-1362-4dce-9789-11b2401500ef None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:40.024833 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 487/969] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Jul 17 16:41:39 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:40.024833 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1dd63-9c60-4bb7-8515-aa703718666f None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:40.024833 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1dd63-9c60-4bb7-8515-aa703718666f None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:40.024833 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1dd63-9c60-4bb7-8515-aa703718666f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:40.046159 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:40.052139 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-03d1dd63-9c60-4bb7-8515-aa703718666f None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:40.054181 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-03d1dd63-9c60-4bb7-8515-aa703718666f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:40.196366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f13d7-eb59-4e2a-8bb5-5cfd520571e2 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:40.216748 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 488/970] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:41:40.242324 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1dd63-9c60-4bb7-8515-aa703718666f None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:40.249203 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:40.276183 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bcc762-5e03-4305-b1f8-b733db34710a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:40.277664 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcc762-5e03-4305-b1f8-b733db34710a None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:40.278339 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcc762-5e03-4305-b1f8-b733db34710a None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:40.279447 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcc762-5e03-4305-b1f8-b733db34710a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:40.418584 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bcc762-5e03-4305-b1f8-b733db34710a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:40.421569 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bcc762-5e03-4305-b1f8-b733db34710a None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:40.475192 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1dd63-9c60-4bb7-8515-aa703718666f None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:40.500765 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:40.568099 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b83af0-0dd1-4967-8919-50aa70572517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:40.569790 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b83af0-0dd1-4967-8919-50aa70572517 None admin] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:40.570637 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b83af0-0dd1-4967-8919-50aa70572517 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:40.571430 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b83af0-0dd1-4967-8919-50aa70572517 None admin] PATH_INFO: `/v3/users` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:40.573037 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b83af0-0dd1-4967-8919-50aa70572517 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:40.576623 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b83af0-0dd1-4967-8919-50aa70572517 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:40.579514 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: WARNING keystone.server.flask.common [None req-88b83af0-0dd1-4967-8919-50aa70572517 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 17 16:41:40.624398 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 483/971] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:41:39 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 661 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 17 16:41:40.633486 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bcc762-5e03-4305-b1f8-b733db34710a None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:40.654584 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.sql.core [None req-88b83af0-0dd1-4967-8919-50aa70572517 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 17 16:41:40.654584 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 17 16:41:40.654584 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [parameters: {'user_id': 'a1e751bd75ae4f6ea5cf00d23311525b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 17 16:41:40.654584 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87198) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 17 16:41:40.699160 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-88b83af0-0dd1-4967-8919-50aa70572517 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 17 16:41:40.699160 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:40.699160 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 17 16:41:40.699160 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 17 16:41:40.699160 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 17 16:41:40.699160 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 17 16:41:40.699160 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 17 16:41:40.699160 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application result = self._query(query) Jul 17 16:41:40.699160 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 17 16:41:40.699160 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application conn.query(q) Jul 17 16:41:40.699160 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 17 16:41:40.699160 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 17 16:41:40.699160 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 17 16:41:40.699160 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application result.read() Jul 17 16:41:40.699160 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 17 16:41:40.700944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 17 16:41:40.700944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 17 16:41:40.700944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 17 16:41:40.700944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 17 16:41:40.700944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 17 16:41:40.700944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 17 16:41:40.700944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 17 16:41:40.700944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 17 16:41:40.700944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:40.700944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 17 16:41:40.700944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:40.700944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:40.700944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 17 16:41:40.700944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 17 16:41:40.700944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 17 16:41:40.700944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 17 16:41:40.700944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 17 16:41:40.700944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application next(self.gen) Jul 17 16:41:40.702863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 17 16:41:40.702863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application yield resource Jul 17 16:41:40.702863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 17 16:41:40.702863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application next(self.gen) Jul 17 16:41:40.702863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 17 16:41:40.702863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application self.session.rollback() Jul 17 16:41:40.702863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 17 16:41:40.702863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application self.force_reraise() Jul 17 16:41:40.702863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 17 16:41:40.702863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise self.value Jul 17 16:41:40.702863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 17 16:41:40.702863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 17 16:41:40.702863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 17 16:41:40.702863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application session.commit() Jul 17 16:41:40.702863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 17 16:41:40.702863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 17 16:41:40.702863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 17 16:41:40.704637 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application self._prepare_impl() Jul 17 16:41:40.704637 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 17 16:41:40.704637 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application self.session.flush() Jul 17 16:41:40.704637 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 17 16:41:40.704637 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application self._flush(objects) Jul 17 16:41:40.704637 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 17 16:41:40.704637 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 17 16:41:40.704637 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 17 16:41:40.704637 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application compat.raise_( Jul 17 16:41:40.704637 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 17 16:41:40.704637 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception Jul 17 16:41:40.704637 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 17 16:41:40.704637 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application flush_context.execute() Jul 17 16:41:40.704637 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 17 16:41:40.704637 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rec.execute(self) Jul 17 16:41:40.704637 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 17 16:41:40.704637 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 17 16:41:40.710761 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 17 16:41:40.710761 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 17 16:41:40.710761 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 17 16:41:40.710761 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 17 16:41:40.710761 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 17 16:41:40.710761 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 17 16:41:40.710761 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 17 16:41:40.710761 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 17 16:41:40.710761 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 17 16:41:40.710761 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 17 16:41:40.710761 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 17 16:41:40.710761 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 17 16:41:40.710761 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 17 16:41:40.710761 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 17 16:41:40.710761 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 17 16:41:40.710761 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception Jul 17 16:41:40.714704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 17 16:41:40.714704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 17 16:41:40.714704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 17 16:41:40.714704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 17 16:41:40.714704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 17 16:41:40.714704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application result = self._query(query) Jul 17 16:41:40.714704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 17 16:41:40.714704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application conn.query(q) Jul 17 16:41:40.714704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 17 16:41:40.714704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 17 16:41:40.714704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 17 16:41:40.714704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application result.read() Jul 17 16:41:40.714704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 17 16:41:40.714704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 17 16:41:40.714704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 17 16:41:40.714704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 17 16:41:40.718829 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 17 16:41:40.718829 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 17 16:41:40.718829 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 17 16:41:40.718829 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 17 16:41:40.718829 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 17 16:41:40.718829 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 17 16:41:40.718829 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [parameters: {'user_id': 'a1e751bd75ae4f6ea5cf00d23311525b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 17 16:41:40.718829 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 17 16:41:40.718829 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:40.718829 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 17 16:41:40.718829 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:40.718829 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:40.718829 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:40.718829 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:40.718829 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:40.722093 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:40.722093 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:40.722093 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:40.722093 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:40.722093 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:40.722093 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:40.722093 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:40.722093 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 17 16:41:40.722093 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 17 16:41:40.722093 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:40.722093 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:40.722093 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:41:40.722093 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:41:40.722093 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:41:40.722093 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:41:40.722093 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 17 16:41:40.722093 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 17 16:41:40.725982 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 17 16:41:40.725982 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 17 16:41:40.725982 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 17 16:41:40.725982 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 17 16:41:40.725982 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 17 16:41:40.725982 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:40.725982 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 489/972] 10.222.0.40 () {60 vars in 1131 bytes} [Sun Jul 17 16:41:40 2022] POST /identity/v3/users => generated 160 bytes in 474 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:41:40.725982 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b83af0-0dd1-4967-8919-50aa70572517 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:40.831347 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bcc762-5e03-4305-b1f8-b733db34710a None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:40.879873 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb922b-4f80-41a2-8e0b-c7d21d2747cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:40.880677 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb922b-4f80-41a2-8e0b-c7d21d2747cb None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:40.880989 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb922b-4f80-41a2-8e0b-c7d21d2747cb None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:40.881278 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb922b-4f80-41a2-8e0b-c7d21d2747cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:40.894491 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:40.899229 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeb922b-4f80-41a2-8e0b-c7d21d2747cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:40.900711 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeb922b-4f80-41a2-8e0b-c7d21d2747cb None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:40.906911 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:40.921462 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-0aeb922b-4f80-41a2-8e0b-c7d21d2747cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:40.921462 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:40.921462 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:40.921462 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:40.921462 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:40.921462 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:40.921462 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:40.921462 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:40.921462 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:40.921462 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:40.921462 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:40.921462 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:40.921462 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:41:40.921462 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:41:40.921462 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:41:40.923500 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:41:40.923500 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:40.923500 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:40.923500 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:40.923500 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:40.923500 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:40.923500 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:40.923500 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:40.923500 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:40.923500 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:40.923500 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:40.923500 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:40.923500 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:40.923500 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:40.923500 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:40.923500 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:40.923500 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:40.925753 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:41:40.925753 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:41:40.925753 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:40.925753 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:40.927590 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 484/973] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:40.928630 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb922b-4f80-41a2-8e0b-c7d21d2747cb None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:40.994594 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b83af0-0dd1-4967-8919-50aa70572517 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:41.056748 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e7687-1298-477a-b8c4-fa0069bf1cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:41.058563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e7687-1298-477a-b8c4-fa0069bf1cab None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:41.059301 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e7687-1298-477a-b8c4-fa0069bf1cab None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:41.061467 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e7687-1298-477a-b8c4-fa0069bf1cab None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:41.075824 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb922b-4f80-41a2-8e0b-c7d21d2747cb None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:41.083391 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e7687-1298-477a-b8c4-fa0069bf1cab None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:41.086186 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e7687-1298-477a-b8c4-fa0069bf1cab None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:41.103459 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-749e7687-1298-477a-b8c4-fa0069bf1cab None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 17 16:41:41.103459 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:41.103459 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:41.103459 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:41.103459 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:41.103459 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:41.103459 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:41.103459 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:41.103459 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:41.103459 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:41.103459 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:41.103459 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:41.103459 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:41:41.103459 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:41:41.103459 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:41:41.103459 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:41:41.105595 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:41.105595 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:41.105595 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:41:41.105595 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:41:41.105595 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:41:41.105595 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:41:41.105595 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:41.105595 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:41.105595 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:41.105595 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:41.105595 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:41.105595 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:41.105595 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:41.105595 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:41.105595 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:41.105595 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:41.105595 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:41.105595 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:41.107758 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:41.107758 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:41.107758 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:41:41.107758 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:41:41.107758 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:41:41.107758 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:41:41.107758 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:41:41.107758 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:41:41.107758 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 17 16:41:41.107758 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:41.107758 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 490/974] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:41:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:41.115312 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e7687-1298-477a-b8c4-fa0069bf1cab None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:41.152699 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4055fb-48b6-42d9-b253-5ddb07e23cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:41.154308 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4055fb-48b6-42d9-b253-5ddb07e23cc2 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:41.154709 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4055fb-48b6-42d9-b253-5ddb07e23cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:41.155026 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4055fb-48b6-42d9-b253-5ddb07e23cc2 None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:41.156183 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4055fb-48b6-42d9-b253-5ddb07e23cc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:41.156599 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4055fb-48b6-42d9-b253-5ddb07e23cc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:41.159405 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4055fb-48b6-42d9-b253-5ddb07e23cc2 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:41.192545 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 485/975] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:41.194587 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4055fb-48b6-42d9-b253-5ddb07e23cc2 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:41.280016 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:41.299715 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e7687-1298-477a-b8c4-fa0069bf1cab None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:41.333559 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:41.345315 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be0dd0-b53e-4b96-8a20-0725f8f1cc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:41.346005 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be0dd0-b53e-4b96-8a20-0725f8f1cc3b None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:41.346288 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be0dd0-b53e-4b96-8a20-0725f8f1cc3b None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:41.346598 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6be0dd0-b53e-4b96-8a20-0725f8f1cc3b None admin] PATH_INFO: `/v3/users` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:41.347376 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6be0dd0-b53e-4b96-8a20-0725f8f1cc3b None admin] RBAC: Adding query filter params (name=cinder) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:41.347678 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6be0dd0-b53e-4b96-8a20-0725f8f1cc3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:41.349349 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6be0dd0-b53e-4b96-8a20-0725f8f1cc3b None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:41.378023 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4055fb-48b6-42d9-b253-5ddb07e23cc2 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:41.401289 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 491/976] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 17 16:41:41 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:41:41.434441 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab2460-61bb-4b19-825d-6dc828288440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:41.436619 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab2460-61bb-4b19-825d-6dc828288440 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:41.437333 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab2460-61bb-4b19-825d-6dc828288440 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:41.437832 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab2460-61bb-4b19-825d-6dc828288440 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:41.448962 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ab2460-61bb-4b19-825d-6dc828288440 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:41.451799 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ab2460-61bb-4b19-825d-6dc828288440 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:41.477110 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-45ab2460-61bb-4b19-825d-6dc828288440 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:41:41.477110 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:41.477110 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:41.477110 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:41.477110 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:41.477110 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:41.477110 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:41.477110 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:41.477110 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:41.477110 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:41.477110 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:41.477110 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:41.477110 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:41:41.477110 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:41:41.477110 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:41:41.478760 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:41:41.478760 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:41.478760 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:41.478760 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:41.478760 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:41.478760 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:41.478760 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:41.478760 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:41.478760 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:41.478760 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:41.478760 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:41.478760 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:41.478760 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:41.478760 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:41.478760 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:41.478760 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:41.478760 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:41.480176 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:41:41.480176 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:41:41.480176 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:41:41.480176 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:41:41.480176 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:41:41.480176 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:41:41.480176 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:41:41.480176 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:41.489155 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 486/977] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:41:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:41:41.494917 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be0dd0-b53e-4b96-8a20-0725f8f1cc3b None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:41.641852 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6be0dd0-b53e-4b96-8a20-0725f8f1cc3b None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:41.691163 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46fdf1a-da6f-463a-b1b6-e3ff18d78538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:41.692577 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46fdf1a-da6f-463a-b1b6-e3ff18d78538 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:41.693120 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46fdf1a-da6f-463a-b1b6-e3ff18d78538 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:41.693603 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46fdf1a-da6f-463a-b1b6-e3ff18d78538 None admin] PATH_INFO: `/v3/projects` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:41.695172 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46fdf1a-da6f-463a-b1b6-e3ff18d78538 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:41.695505 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46fdf1a-da6f-463a-b1b6-e3ff18d78538 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:41.698325 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46fdf1a-da6f-463a-b1b6-e3ff18d78538 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:41.722764 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 492/978] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:41:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:41.729714 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab2460-61bb-4b19-825d-6dc828288440 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:41.858213 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab2460-61bb-4b19-825d-6dc828288440 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:41.870114 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:41.897122 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88937faa-6c83-4350-9e1b-858f3cde6c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:41.897723 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88937faa-6c83-4350-9e1b-858f3cde6c9c None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:41.898018 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88937faa-6c83-4350-9e1b-858f3cde6c9c None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:41.898303 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88937faa-6c83-4350-9e1b-858f3cde6c9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:41.899152 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88937faa-6c83-4350-9e1b-858f3cde6c9c None admin] RBAC: Adding query filter params (role.id=68d3a95a9fc34fd1a28a9861d22ee828, scope.project.id=927057f9a5af4833aeac9992203a223c, user.id=37ed5f72dc8c4a5685bd1245cadcdf7e) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:41.899471 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88937faa-6c83-4350-9e1b-858f3cde6c9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:41.900957 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88937faa-6c83-4350-9e1b-858f3cde6c9c None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:41.910218 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 487/979] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:41:41 2022] GET /identity/v3/role_assignments?user.id=37ed5f72dc8c4a5685bd1245cadcdf7e&scope.project.id=927057f9a5af4833aeac9992203a223c&role.id=68d3a95a9fc34fd1a28a9861d22ee828 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:42.168955 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3541fc9-9ee9-4e57-a9eb-25da819cc609 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:42.168955 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3541fc9-9ee9-4e57-a9eb-25da819cc609 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:42.168955 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3541fc9-9ee9-4e57-a9eb-25da819cc609 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:42.169700 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 493/980] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:42.175472 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61273c6b-074a-4370-ac26-418d8d9ded49 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:42.175751 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61273c6b-074a-4370-ac26-418d8d9ded49 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:42.176110 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61273c6b-074a-4370-ac26-418d8d9ded49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:42.195187 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-61273c6b-074a-4370-ac26-418d8d9ded49 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:42.196419 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-61273c6b-074a-4370-ac26-418d8d9ded49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:42.287072 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 488/981] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 17 16:41:42.310364 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3541fc9-9ee9-4e57-a9eb-25da819cc609 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:42.366222 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:42.439263 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f24b61-5708-45ff-8742-b123eb7fb31d None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:42.439626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f24b61-5708-45ff-8742-b123eb7fb31d None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:42.440128 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f24b61-5708-45ff-8742-b123eb7fb31d None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:42.442390 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3541fc9-9ee9-4e57-a9eb-25da819cc609 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:42.443104 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 489/982] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:42.449802 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5da14-f2c4-4c10-adbf-a83fddb9eb24 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:42.450068 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5da14-f2c4-4c10-adbf-a83fddb9eb24 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:42.451066 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5da14-f2c4-4c10-adbf-a83fddb9eb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:42.473458 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-81e5da14-f2c4-4c10-adbf-a83fddb9eb24 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:42.475743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-81e5da14-f2c4-4c10-adbf-a83fddb9eb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:42.489219 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab0d40-abe4-4ecb-8b2a-01021d0a6835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:42.489795 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab0d40-abe4-4ecb-8b2a-01021d0a6835 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:42.490079 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab0d40-abe4-4ecb-8b2a-01021d0a6835 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:42.490360 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab0d40-abe4-4ecb-8b2a-01021d0a6835 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:42.491028 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab0d40-abe4-4ecb-8b2a-01021d0a6835 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:42.492974 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab0d40-abe4-4ecb-8b2a-01021d0a6835 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:42.509303 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-a1ab0d40-abe4-4ecb-8b2a-01021d0a6835 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 17 16:41:42.509303 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:42.509303 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 17 16:41:42.509303 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 17 16:41:42.509303 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 17 16:41:42.509303 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 17 16:41:42.509303 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 17 16:41:42.509303 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:42.509303 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 17 16:41:42.509303 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:42.509303 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 17 16:41:42.509303 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:42.509303 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:42.509303 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:42.509303 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:42.509303 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:42.509303 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:42.510646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:42.510646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:42.510646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:42.510646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:42.510646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:42.510646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:42.510646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 17 16:41:42.510646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 17 16:41:42.510646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 17 16:41:42.510646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 17 16:41:42.510646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:42.510646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:42.510646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:42.510646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:42.510646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:42.510646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:42.511527 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:42.511527 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:42.511527 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:42.511527 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:42.511527 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:42.511527 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:42.511527 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:42.511527 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:42.511527 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:42.511527 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:42.511527 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 17 16:41:42.511527 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:42.511527 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 17 16:41:42.511527 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:42.519410 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 494/983] 10.222.0.40 () {56 vars in 1126 bytes} [Sun Jul 17 16:41:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:42.526252 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab0d40-abe4-4ecb-8b2a-01021d0a6835 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:42.579948 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 490/984] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 17 16:41:42.621152 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5da14-f2c4-4c10-adbf-a83fddb9eb24 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:42.677985 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:42.688638 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab0d40-abe4-4ecb-8b2a-01021d0a6835 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:42.702211 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:42.741512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a5166-7bcf-4a66-90fc-8eb37f6271a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:42.742574 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a5166-7bcf-4a66-90fc-8eb37f6271a6 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:42.743796 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a5166-7bcf-4a66-90fc-8eb37f6271a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:42.744269 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17a5166-7bcf-4a66-90fc-8eb37f6271a6 None admin] PATH_INFO: `/v3/services` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:42.745563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17a5166-7bcf-4a66-90fc-8eb37f6271a6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:42.746065 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17a5166-7bcf-4a66-90fc-8eb37f6271a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:42.749735 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17a5166-7bcf-4a66-90fc-8eb37f6271a6 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:42.759044 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 495/985] 10.222.0.40 () {56 vars in 1122 bytes} [Sun Jul 17 16:41:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:42.767590 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a5166-7bcf-4a66-90fc-8eb37f6271a6 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:42.788640 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5da14-f2c4-4c10-adbf-a83fddb9eb24 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:42.843575 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb54ac1-80f5-40be-aa07-adfebcfbd9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:42.844261 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb54ac1-80f5-40be-aa07-adfebcfbd9b6 None admin] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:42.844556 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb54ac1-80f5-40be-aa07-adfebcfbd9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:42.845416 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb54ac1-80f5-40be-aa07-adfebcfbd9b6 None admin] PATH_INFO: `/v3/services` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:42.846160 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb54ac1-80f5-40be-aa07-adfebcfbd9b6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:42.847873 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb54ac1-80f5-40be-aa07-adfebcfbd9b6 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:42.902035 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17a5166-7bcf-4a66-90fc-8eb37f6271a6 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:42.954469 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 491/986] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Jul 17 16:41:42 2022] POST /identity/v3/services => generated 242 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 17 16:41:42.973684 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81ff4c-00b1-4abf-b013-10dae38cb267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:42.974479 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81ff4c-00b1-4abf-b013-10dae38cb267 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:42.974760 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81ff4c-00b1-4abf-b013-10dae38cb267 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:42.975035 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81ff4c-00b1-4abf-b013-10dae38cb267 None admin] PATH_INFO: `/v3/services` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:42.975788 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb81ff4c-00b1-4abf-b013-10dae38cb267 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:42.976117 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb81ff4c-00b1-4abf-b013-10dae38cb267 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:42.977601 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb81ff4c-00b1-4abf-b013-10dae38cb267 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:42.984957 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 496/987] 10.222.0.40 () {56 vars in 1122 bytes} [Sun Jul 17 16:41:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:42.990974 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb54ac1-80f5-40be-aa07-adfebcfbd9b6 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:43.121414 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:43.140081 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb54ac1-80f5-40be-aa07-adfebcfbd9b6 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:43.199605 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d7d0e9-f87b-42c0-a9dd-3d114e704c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:43.201643 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7d0e9-f87b-42c0-a9dd-3d114e704c32 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:43.202181 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7d0e9-f87b-42c0-a9dd-3d114e704c32 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:43.202945 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d7d0e9-f87b-42c0-a9dd-3d114e704c32 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:43.204067 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d7d0e9-f87b-42c0-a9dd-3d114e704c32 None admin] RBAC: Adding query filter params (interface=admin, service_id=2151f51760474456a8080af98b21070a, region_id=RegionOne) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:43.204410 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d7d0e9-f87b-42c0-a9dd-3d114e704c32 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:43.206278 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d7d0e9-f87b-42c0-a9dd-3d114e704c32 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:43.213183 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 492/988] 10.222.0.40 () {56 vars in 1249 bytes} [Sun Jul 17 16:41:42 2022] GET /identity/v3/endpoints?service_id=2151f51760474456a8080af98b21070a&interface=admin®ion_id=RegionOne => generated 193 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:43.220432 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81ff4c-00b1-4abf-b013-10dae38cb267 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:43.257830 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:43.374329 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81ff4c-00b1-4abf-b013-10dae38cb267 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:43.412910 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a033b3-5e9d-426f-8369-8da9a4c58532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:43.413589 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a033b3-5e9d-426f-8369-8da9a4c58532 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:43.413874 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a033b3-5e9d-426f-8369-8da9a4c58532 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:43.414138 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a033b3-5e9d-426f-8369-8da9a4c58532 None admin] PATH_INFO: `/v3/services` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:43.414818 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a033b3-5e9d-426f-8369-8da9a4c58532 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:43.416204 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a033b3-5e9d-426f-8369-8da9a4c58532 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:43.434367 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 497/989] 10.222.0.40 () {56 vars in 1087 bytes} [Sun Jul 17 16:41:43 2022] GET /identity/v3/services => generated 1727 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 17 16:41:44.136503 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71225e2-4813-48b7-bc2b-d1fe1d7aa9f6 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:44.137361 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71225e2-4813-48b7-bc2b-d1fe1d7aa9f6 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:44.137361 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71225e2-4813-48b7-bc2b-d1fe1d7aa9f6 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:44.139314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 493/990] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:41:44.145768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691dc018-5ccc-46b5-87d5-2677c8a8715b None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:44.146407 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691dc018-5ccc-46b5-87d5-2677c8a8715b None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:44.147003 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691dc018-5ccc-46b5-87d5-2677c8a8715b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:44.177444 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-691dc018-5ccc-46b5-87d5-2677c8a8715b None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:44.180503 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-691dc018-5ccc-46b5-87d5-2677c8a8715b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:44.328821 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 498/991] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:44 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 17 16:41:44.365550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71225e2-4813-48b7-bc2b-d1fe1d7aa9f6 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:44.561037 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71225e2-4813-48b7-bc2b-d1fe1d7aa9f6 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:44.628721 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee9051a-f814-4472-be81-e22327506a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:44.630147 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9051a-f814-4472-be81-e22327506a02 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:44.630654 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9051a-f814-4472-be81-e22327506a02 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:44.631154 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9051a-f814-4472-be81-e22327506a02 None admin] PATH_INFO: `/v3/services/image` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:44.632399 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee9051a-f814-4472-be81-e22327506a02 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:44.635119 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee9051a-f814-4472-be81-e22327506a02 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:44.659817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-9ee9051a-f814-4472-be81-e22327506a02 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 17 16:41:44.659817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:44.659817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 17 16:41:44.659817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 17 16:41:44.659817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 17 16:41:44.659817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 17 16:41:44.659817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 17 16:41:44.659817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:44.659817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 17 16:41:44.659817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:44.659817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 17 16:41:44.659817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:44.659817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:44.659817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:44.659817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:44.659817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:44.659817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:44.660948 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:44.660948 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:44.660948 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:44.660948 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:44.660948 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:44.660948 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:44.660948 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 17 16:41:44.660948 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 17 16:41:44.660948 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 17 16:41:44.660948 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 17 16:41:44.660948 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:44.660948 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:44.660948 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:44.660948 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:44.660948 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:44.660948 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:44.661790 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:44.661790 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:44.661790 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:44.661790 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:44.661790 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:44.661790 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:44.661790 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:44.661790 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:44.661790 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:44.661790 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:44.661790 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 17 16:41:44.661790 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:44.661790 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 17 16:41:44.661790 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:44.662539 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 494/992] 10.222.0.40 () {56 vars in 1105 bytes} [Sun Jul 17 16:41:44 2022] GET /identity/v3/services/image => generated 86 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:41:44.672910 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691dc018-5ccc-46b5-87d5-2677c8a8715b None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:44.730478 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:44.868298 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691dc018-5ccc-46b5-87d5-2677c8a8715b None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:44.933290 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d2c9e5-7630-4e8c-b333-6cec77561101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:44.934600 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2c9e5-7630-4e8c-b333-6cec77561101 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:44.935114 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2c9e5-7630-4e8c-b333-6cec77561101 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:44.935614 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d2c9e5-7630-4e8c-b333-6cec77561101 None admin] PATH_INFO: `/v3/services` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:44.937001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2c9e5-7630-4e8c-b333-6cec77561101 None admin] RBAC: Adding query filter params (name=image) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:44.937563 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2c9e5-7630-4e8c-b333-6cec77561101 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:44.940222 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d2c9e5-7630-4e8c-b333-6cec77561101 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:44.957172 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 499/993] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:44 2022] GET /identity/v3/services?name=image => generated 122 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:44.960154 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee9051a-f814-4472-be81-e22327506a02 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:44.998231 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:45.121735 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed73aba2-0b3d-4274-92df-f2d18bf598f1 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:45.122648 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed73aba2-0b3d-4274-92df-f2d18bf598f1 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:45.122789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed73aba2-0b3d-4274-92df-f2d18bf598f1 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:45.123953 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee9051a-f814-4472-be81-e22327506a02 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:45.124612 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 500/994] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:41:45.130004 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e465d67-2b1c-40ad-8290-5b57ba1bc03c None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:45.130368 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e465d67-2b1c-40ad-8290-5b57ba1bc03c None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:45.130555 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e465d67-2b1c-40ad-8290-5b57ba1bc03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:45.149025 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-2e465d67-2b1c-40ad-8290-5b57ba1bc03c None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:45.150263 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-2e465d67-2b1c-40ad-8290-5b57ba1bc03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:45.196978 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211fbf01-2414-4f21-a754-291f4aa52eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:45.198135 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211fbf01-2414-4f21-a754-291f4aa52eef None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:45.198642 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211fbf01-2414-4f21-a754-291f4aa52eef None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:45.199143 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211fbf01-2414-4f21-a754-291f4aa52eef None admin] PATH_INFO: `/v3/services` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:45.201096 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211fbf01-2414-4f21-a754-291f4aa52eef None admin] RBAC: Adding query filter params (type=image) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:45.201655 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211fbf01-2414-4f21-a754-291f4aa52eef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:45.204257 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211fbf01-2414-4f21-a754-291f4aa52eef None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:45.212065 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 495/995] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:44 2022] GET /identity/v3/services?type=image => generated 122 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:45.271237 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 501/996] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:45 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 17 16:41:45.295069 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211fbf01-2414-4f21-a754-291f4aa52eef None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:45.420818 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:45.448798 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211fbf01-2414-4f21-a754-291f4aa52eef None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:45.513280 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df15087-4e18-43ed-b552-a1c4e49ae667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:45.514664 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df15087-4e18-43ed-b552-a1c4e49ae667 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:45.515167 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df15087-4e18-43ed-b552-a1c4e49ae667 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:45.516121 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df15087-4e18-43ed-b552-a1c4e49ae667 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:45.517378 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df15087-4e18-43ed-b552-a1c4e49ae667 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:45.519998 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df15087-4e18-43ed-b552-a1c4e49ae667 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:45.543363 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-3df15087-4e18-43ed-b552-a1c4e49ae667 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 17 16:41:45.543363 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:45.543363 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 17 16:41:45.543363 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 17 16:41:45.543363 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 17 16:41:45.543363 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 17 16:41:45.543363 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 17 16:41:45.543363 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:45.543363 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 17 16:41:45.543363 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:45.543363 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 17 16:41:45.543363 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:45.543363 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:45.543363 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:45.543363 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:45.543363 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:45.543363 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:45.545252 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:45.545252 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:45.545252 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:45.545252 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:45.545252 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:45.545252 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:45.545252 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 17 16:41:45.545252 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 17 16:41:45.545252 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 17 16:41:45.545252 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 17 16:41:45.545252 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:45.545252 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:45.545252 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:45.545252 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:45.545252 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:45.545252 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:45.546101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:45.546101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:45.546101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:45.546101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:45.546101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:45.546101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:45.546101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:45.546101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:45.546101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:45.546101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:45.546101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 17 16:41:45.546101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:45.546101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 17 16:41:45.546101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:45.546836 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 496/997] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:41:45 2022] GET /identity/v3/services/compute => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:45.553622 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e465d67-2b1c-40ad-8290-5b57ba1bc03c None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:45.591957 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:45.734638 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e465d67-2b1c-40ad-8290-5b57ba1bc03c None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:45.796800 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5d2e3-f632-4fe2-836b-b9b825ef52b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:45.797974 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5d2e3-f632-4fe2-836b-b9b825ef52b6 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:45.798472 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5d2e3-f632-4fe2-836b-b9b825ef52b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:45.798962 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5d2e3-f632-4fe2-836b-b9b825ef52b6 None admin] PATH_INFO: `/v3/services` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:45.800231 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c5d2e3-f632-4fe2-836b-b9b825ef52b6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:45.800777 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c5d2e3-f632-4fe2-836b-b9b825ef52b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:45.803395 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c5d2e3-f632-4fe2-836b-b9b825ef52b6 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:45.817752 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 502/998] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Jul 17 16:41:45 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:45.819632 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df15087-4e18-43ed-b552-a1c4e49ae667 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:45.851093 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9460b7d-9e40-4841-8d6a-0983ad9f43b3 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:45.851423 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9460b7d-9e40-4841-8d6a-0983ad9f43b3 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:45.851667 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9460b7d-9e40-4841-8d6a-0983ad9f43b3 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:45.853249 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 503/999] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:41:45.859388 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b97ec5c-7e3c-4ce9-ab90-1c7128b68ea0 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:45.859685 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b97ec5c-7e3c-4ce9-ab90-1c7128b68ea0 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:45.860047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b97ec5c-7e3c-4ce9-ab90-1c7128b68ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:45.878850 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-2b97ec5c-7e3c-4ce9-ab90-1c7128b68ea0 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:45.880063 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-2b97ec5c-7e3c-4ce9-ab90-1c7128b68ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:45.959899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 504/1000] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:45 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 17 16:41:45.966443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df15087-4e18-43ed-b552-a1c4e49ae667 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:45.983753 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b97ec5c-7e3c-4ce9-ab90-1c7128b68ea0 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:46.001148 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:46.009944 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1435e0f-294e-4205-8a85-342f5e71a45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:46.010625 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1435e0f-294e-4205-8a85-342f5e71a45b None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:46.010904 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1435e0f-294e-4205-8a85-342f5e71a45b None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:46.011187 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1435e0f-294e-4205-8a85-342f5e71a45b None admin] PATH_INFO: `/v3/services` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:46.011903 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1435e0f-294e-4205-8a85-342f5e71a45b None admin] RBAC: Adding query filter params (type=compute) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:46.012219 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1435e0f-294e-4205-8a85-342f5e71a45b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:46.014679 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1435e0f-294e-4205-8a85-342f5e71a45b None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:46.020907 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 497/1001] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Jul 17 16:41:45 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:46.026922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1435e0f-294e-4205-8a85-342f5e71a45b None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:46.046673 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:46.105714 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b97ec5c-7e3c-4ce9-ab90-1c7128b68ea0 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:46.147502 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4bc934-d250-4c9d-adf9-e7d1131e6b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:46.148321 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bc934-d250-4c9d-adf9-e7d1131e6b33 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:46.148605 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bc934-d250-4c9d-adf9-e7d1131e6b33 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:46.148913 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bc934-d250-4c9d-adf9-e7d1131e6b33 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:46.149598 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4bc934-d250-4c9d-adf9-e7d1131e6b33 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:46.150986 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4bc934-d250-4c9d-adf9-e7d1131e6b33 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:46.157631 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1435e0f-294e-4205-8a85-342f5e71a45b None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:46.165192 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-6e4bc934-d250-4c9d-adf9-e7d1131e6b33 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 17 16:41:46.165192 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:46.165192 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 17 16:41:46.165192 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 17 16:41:46.165192 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 17 16:41:46.165192 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 17 16:41:46.165192 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 17 16:41:46.165192 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:46.165192 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 17 16:41:46.165192 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:46.165192 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 17 16:41:46.165192 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:46.165192 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:46.165192 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:46.165192 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:46.165192 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:46.165192 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:46.166242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:46.166242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:46.166242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:46.166242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:46.166242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:46.166242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:46.166242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 17 16:41:46.166242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 17 16:41:46.166242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 17 16:41:46.166242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 17 16:41:46.166242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:46.166242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:46.166242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:46.166242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:46.166242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:46.166242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:46.167648 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:46.167648 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:46.167648 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:46.167648 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:46.167648 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:46.167648 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:46.167648 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:46.167648 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:46.167648 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:46.167648 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:46.167648 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 17 16:41:46.167648 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:46.167648 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 17 16:41:46.167648 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:46.167648 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 505/1002] 10.222.0.40 () {56 vars in 1126 bytes} [Sun Jul 17 16:41:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:41:46.178491 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4bc934-d250-4c9d-adf9-e7d1131e6b33 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:46.214620 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ce63f-ec42-4c93-9ec6-1594f4b3116b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:46.216198 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ce63f-ec42-4c93-9ec6-1594f4b3116b None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:46.216320 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ce63f-ec42-4c93-9ec6-1594f4b3116b None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:46.217508 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ce63f-ec42-4c93-9ec6-1594f4b3116b None admin] PATH_INFO: `/v3/endpoints` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:46.218278 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ce63f-ec42-4c93-9ec6-1594f4b3116b None admin] RBAC: Adding query filter params (interface=public, service_id=b8813203e46c48cca9f2bb97c1dcc265, region_id=RegionOne) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:46.218652 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ce63f-ec42-4c93-9ec6-1594f4b3116b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:46.220056 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ce63f-ec42-4c93-9ec6-1594f4b3116b None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:46.226208 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 498/1003] 10.222.0.40 () {56 vars in 1251 bytes} [Sun Jul 17 16:41:46 2022] GET /identity/v3/endpoints?service_id=b8813203e46c48cca9f2bb97c1dcc265&interface=public®ion_id=RegionOne => generated 194 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:41:46.232513 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ce63f-ec42-4c93-9ec6-1594f4b3116b None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:46.281712 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:46.316066 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4bc934-d250-4c9d-adf9-e7d1131e6b33 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:46.354401 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:46.375989 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab694b5-9464-4065-88f1-dcc94a37ca26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:46.377120 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab694b5-9464-4065-88f1-dcc94a37ca26 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:46.377594 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab694b5-9464-4065-88f1-dcc94a37ca26 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:46.378039 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab694b5-9464-4065-88f1-dcc94a37ca26 None admin] PATH_INFO: `/v3/services` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:46.378959 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab694b5-9464-4065-88f1-dcc94a37ca26 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:46.379431 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab694b5-9464-4065-88f1-dcc94a37ca26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:46.381165 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab694b5-9464-4065-88f1-dcc94a37ca26 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:46.393597 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 506/1004] 10.222.0.40 () {56 vars in 1122 bytes} [Sun Jul 17 16:41:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:46.396049 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab694b5-9464-4065-88f1-dcc94a37ca26 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:46.397920 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ce63f-ec42-4c93-9ec6-1594f4b3116b None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:46.465175 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6dce69-6445-47ee-b0db-4f6754ea39d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:46.466985 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6dce69-6445-47ee-b0db-4f6754ea39d1 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:46.467469 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6dce69-6445-47ee-b0db-4f6754ea39d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:46.467928 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6dce69-6445-47ee-b0db-4f6754ea39d1 None admin] PATH_INFO: `/v3/services` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:46.469128 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6dce69-6445-47ee-b0db-4f6754ea39d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:46.471518 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6dce69-6445-47ee-b0db-4f6754ea39d1 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:46.483732 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 499/1005] 10.222.0.40 () {56 vars in 1087 bytes} [Sun Jul 17 16:41:46 2022] GET /identity/v3/services => generated 1727 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 17 16:41:46.588549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab694b5-9464-4065-88f1-dcc94a37ca26 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:46.638202 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66616229-6245-4d57-8fdf-fd6591e11f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:46.639169 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66616229-6245-4d57-8fdf-fd6591e11f1a None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:46.639585 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66616229-6245-4d57-8fdf-fd6591e11f1a None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:46.639997 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66616229-6245-4d57-8fdf-fd6591e11f1a None admin] PATH_INFO: `/v3/services` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:46.641035 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66616229-6245-4d57-8fdf-fd6591e11f1a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:46.641584 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66616229-6245-4d57-8fdf-fd6591e11f1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:46.643377 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66616229-6245-4d57-8fdf-fd6591e11f1a None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:46.649874 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 507/1006] 10.222.0.40 () {56 vars in 1122 bytes} [Sun Jul 17 16:41:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:46.656439 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6dce69-6445-47ee-b0db-4f6754ea39d1 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:46.823878 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6dce69-6445-47ee-b0db-4f6754ea39d1 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:46.891411 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66455ce7-9156-45d4-ace4-0e0b6b038ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:46.892205 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66455ce7-9156-45d4-ace4-0e0b6b038ea4 None admin] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:46.892611 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66455ce7-9156-45d4-ace4-0e0b6b038ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:46.893052 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66455ce7-9156-45d4-ace4-0e0b6b038ea4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:46.893945 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66455ce7-9156-45d4-ace4-0e0b6b038ea4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:46.895458 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66455ce7-9156-45d4-ace4-0e0b6b038ea4 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:46.974420 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 500/1007] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 17 16:41:46 2022] POST /identity/v3/endpoints => generated 325 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 17 16:41:48.344208 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f52165-ba27-438a-b97e-644a6b6f8531 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:48.345054 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f52165-ba27-438a-b97e-644a6b6f8531 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:48.345233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f52165-ba27-438a-b97e-644a6b6f8531 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:48.348619 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 508/1008] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:48.358043 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1455aa-6133-4f99-ab0f-6f168f68a716 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:48.358788 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1455aa-6133-4f99-ab0f-6f168f68a716 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:48.359250 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1455aa-6133-4f99-ab0f-6f168f68a716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:48.367993 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:48.391034 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-4f1455aa-6133-4f99-ab0f-6f168f68a716 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:48.393352 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-4f1455aa-6133-4f99-ab0f-6f168f68a716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:48.545303 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 501/1009] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:48 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:41:48.578501 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f52165-ba27-438a-b97e-644a6b6f8531 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:48.596645 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:48.654290 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a994a53-8512-479c-8946-927d4e348dd2 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:48.656146 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a994a53-8512-479c-8946-927d4e348dd2 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:48.656632 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a994a53-8512-479c-8946-927d4e348dd2 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:48.659714 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 502/1010] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:48.668860 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a84a32-c296-4043-87c7-344081017f5d None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:48.668860 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a84a32-c296-4043-87c7-344081017f5d None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:48.669699 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a84a32-c296-4043-87c7-344081017f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:48.708216 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-d9a84a32-c296-4043-87c7-344081017f5d None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:48.710488 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-d9a84a32-c296-4043-87c7-344081017f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:48.749154 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f52165-ba27-438a-b97e-644a6b6f8531 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:48.788920 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a1fb4-4ce0-48aa-a5bb-3ceaa5edf9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:48.789953 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a1fb4-4ce0-48aa-a5bb-3ceaa5edf9c6 None admin] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:48.790428 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a1fb4-4ce0-48aa-a5bb-3ceaa5edf9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:48.790881 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a1fb4-4ce0-48aa-a5bb-3ceaa5edf9c6 None admin] PATH_INFO: `/v3/services` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:48.791794 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0a1fb4-4ce0-48aa-a5bb-3ceaa5edf9c6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:48.793402 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0a1fb4-4ce0-48aa-a5bb-3ceaa5edf9c6 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:48.842020 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 503/1011] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:48 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:41:48.865218 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 509/1012] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Jul 17 16:41:48 2022] POST /identity/v3/services => generated 242 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:41:48.865571 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a84a32-c296-4043-87c7-344081017f5d None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:48.946998 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:49.019720 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a84a32-c296-4043-87c7-344081017f5d None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:49.096968 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a6f3ae-84ca-45f8-8cc0-0c966648c467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:49.098275 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6f3ae-84ca-45f8-8cc0-0c966648c467 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:49.098769 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6f3ae-84ca-45f8-8cc0-0c966648c467 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:49.099259 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6f3ae-84ca-45f8-8cc0-0c966648c467 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:49.100580 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a6f3ae-84ca-45f8-8cc0-0c966648c467 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:49.103240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a6f3ae-84ca-45f8-8cc0-0c966648c467 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:49.127933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-c6a6f3ae-84ca-45f8-8cc0-0c966648c467 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 17 16:41:49.127933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:49.127933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 17 16:41:49.127933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 17 16:41:49.127933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 17 16:41:49.127933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 17 16:41:49.127933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 17 16:41:49.127933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:49.127933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 17 16:41:49.127933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:49.127933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 17 16:41:49.127933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:49.127933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:49.127933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:49.127933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:49.127933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:49.127933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:49.129278 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:49.129278 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:49.129278 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:49.129278 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:49.129278 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:49.129278 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:49.129278 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 17 16:41:49.129278 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 17 16:41:49.129278 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 17 16:41:49.129278 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 17 16:41:49.129278 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:49.129278 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:49.129278 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:49.129278 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:49.129278 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:49.129278 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:49.130138 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:49.130138 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:49.130138 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:49.130138 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:49.130138 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:49.130138 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:49.130138 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:49.130138 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:49.130138 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:49.130138 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:49.130138 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 17 16:41:49.130138 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:49.130138 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 17 16:41:49.130138 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:49.131541 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 504/1013] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:41:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:41:49.137058 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a1fb4-4ce0-48aa-a5bb-3ceaa5edf9c6 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:49.222503 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:49.264695 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a1fb4-4ce0-48aa-a5bb-3ceaa5edf9c6 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:49.304231 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4638b0-1424-4ab1-9402-9ac4d4edb8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:49.304950 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4638b0-1424-4ab1-9402-9ac4d4edb8a3 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:49.305241 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4638b0-1424-4ab1-9402-9ac4d4edb8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:49.305518 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4638b0-1424-4ab1-9402-9ac4d4edb8a3 None admin] PATH_INFO: `/v3/services` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:49.306238 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4638b0-1424-4ab1-9402-9ac4d4edb8a3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:49.306542 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4638b0-1424-4ab1-9402-9ac4d4edb8a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:49.307950 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4638b0-1424-4ab1-9402-9ac4d4edb8a3 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:49.314287 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 510/1014] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Jul 17 16:41:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:49.320454 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a6f3ae-84ca-45f8-8cc0-0c966648c467 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:49.349400 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc7714-abd3-410d-84e6-4deb1be63429 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:49.349924 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc7714-abd3-410d-84e6-4deb1be63429 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:49.349924 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc7714-abd3-410d-84e6-4deb1be63429 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:49.351278 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 511/1015] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:49.356896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3debd7-1703-44ef-9f1b-1eed552b2f53 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:49.357198 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3debd7-1703-44ef-9f1b-1eed552b2f53 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:49.357610 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3debd7-1703-44ef-9f1b-1eed552b2f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:49.378530 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-1e3debd7-1703-44ef-9f1b-1eed552b2f53 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:49.380096 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-1e3debd7-1703-44ef-9f1b-1eed552b2f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:49.457248 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a6f3ae-84ca-45f8-8cc0-0c966648c467 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:49.469219 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 512/1016] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:49 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 17 16:41:49.484382 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3debd7-1703-44ef-9f1b-1eed552b2f53 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:49.490019 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:49.521320 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745bbc4-a515-4d79-8f88-7efdc8d834ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:49.522539 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745bbc4-a515-4d79-8f88-7efdc8d834ea None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:49.523239 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745bbc4-a515-4d79-8f88-7efdc8d834ea None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:49.524967 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745bbc4-a515-4d79-8f88-7efdc8d834ea None admin] PATH_INFO: `/v3/services` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:49.526397 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6745bbc4-a515-4d79-8f88-7efdc8d834ea None admin] RBAC: Adding query filter params (type=compute) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:49.526967 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6745bbc4-a515-4d79-8f88-7efdc8d834ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:49.530589 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6745bbc4-a515-4d79-8f88-7efdc8d834ea None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:49.542665 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 505/1017] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Jul 17 16:41:49 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:49.549247 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745bbc4-a515-4d79-8f88-7efdc8d834ea None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:49.557115 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:49.626746 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3debd7-1703-44ef-9f1b-1eed552b2f53 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:49.688007 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf9a51a-a096-4e30-9ac5-5010b74530de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:49.689541 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf9a51a-a096-4e30-9ac5-5010b74530de None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:49.690050 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf9a51a-a096-4e30-9ac5-5010b74530de None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:49.690511 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf9a51a-a096-4e30-9ac5-5010b74530de None admin] PATH_INFO: `/v3/domains/default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:49.692384 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf9a51a-a096-4e30-9ac5-5010b74530de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:49.694333 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf9a51a-a096-4e30-9ac5-5010b74530de None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:49.696482 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 513/1018] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:41:49.704152 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf9a51a-a096-4e30-9ac5-5010b74530de None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:49.705407 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745bbc4-a515-4d79-8f88-7efdc8d834ea None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:49.750199 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd3818-7107-4533-a3d5-f1c81705f0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:49.750873 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd3818-7107-4533-a3d5-f1c81705f0b6 None admin] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:49.751161 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd3818-7107-4533-a3d5-f1c81705f0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:49.751445 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd3818-7107-4533-a3d5-f1c81705f0b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:49.752390 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd3818-7107-4533-a3d5-f1c81705f0b6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:49.753842 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd3818-7107-4533-a3d5-f1c81705f0b6 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:49.844244 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf9a51a-a096-4e30-9ac5-5010b74530de None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:49.874765 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 506/1019] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 17 16:41:49 2022] POST /identity/v3/endpoints => generated 334 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 17 16:41:49.898206 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:49.923649 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee063ea-36c7-4d8f-91f5-2a6b0c72db63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:49.924962 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee063ea-36c7-4d8f-91f5-2a6b0c72db63 None admin] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:49.925433 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee063ea-36c7-4d8f-91f5-2a6b0c72db63 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:49.925933 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee063ea-36c7-4d8f-91f5-2a6b0c72db63 None admin] PATH_INFO: `/v3/projects` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:49.927309 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee063ea-36c7-4d8f-91f5-2a6b0c72db63 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:49.930185 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee063ea-36c7-4d8f-91f5-2a6b0c72db63 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:50.034812 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 514/1020] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Jul 17 16:41:49 2022] POST /identity/v3/projects => generated 311 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:41:51.229096 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189f985-db5d-46cf-aea7-c7e07eeda468 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:51.229881 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189f985-db5d-46cf-aea7-c7e07eeda468 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:51.229881 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189f985-db5d-46cf-aea7-c7e07eeda468 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:51.232836 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 507/1021] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:51.239942 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11633966-e8e5-4037-ba70-e05b3658e6a3 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:51.240546 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11633966-e8e5-4037-ba70-e05b3658e6a3 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:51.241005 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11633966-e8e5-4037-ba70-e05b3658e6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:51.268701 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-11633966-e8e5-4037-ba70-e05b3658e6a3 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:51.269997 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-11633966-e8e5-4037-ba70-e05b3658e6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:51.350272 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 515/1022] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:51 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:41:51.374352 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189f985-db5d-46cf-aea7-c7e07eeda468 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:51.438185 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:51.514736 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189f985-db5d-46cf-aea7-c7e07eeda468 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:51.557633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ef45db-3849-4916-a8ca-f3e91864b178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:51.558456 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ef45db-3849-4916-a8ca-f3e91864b178 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:51.558741 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ef45db-3849-4916-a8ca-f3e91864b178 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:51.559149 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ef45db-3849-4916-a8ca-f3e91864b178 None admin] PATH_INFO: `/v3/services/image` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:51.559771 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ef45db-3849-4916-a8ca-f3e91864b178 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:51.561209 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ef45db-3849-4916-a8ca-f3e91864b178 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:51.573922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-30ef45db-3849-4916-a8ca-f3e91864b178 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 17 16:41:51.573922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:51.573922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 17 16:41:51.573922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 17 16:41:51.573922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 17 16:41:51.573922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 17 16:41:51.573922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 17 16:41:51.573922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:51.573922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 17 16:41:51.573922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:51.573922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 17 16:41:51.573922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:51.573922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:51.573922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:51.573922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:51.573922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:51.573922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:51.575539 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:51.575539 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:51.575539 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:51.575539 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:51.575539 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:51.575539 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:51.575539 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 17 16:41:51.575539 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 17 16:41:51.575539 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 17 16:41:51.575539 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 17 16:41:51.575539 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:51.575539 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:51.575539 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:51.575539 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:51.575539 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:51.575539 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:51.576377 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:51.576377 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:51.576377 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:51.576377 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:51.576377 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:51.576377 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:51.576377 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:51.576377 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:51.576377 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:51.576377 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:51.576377 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 17 16:41:51.576377 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:51.576377 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 17 16:41:51.576377 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:51.576377 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 508/1023] 10.222.0.40 () {56 vars in 1105 bytes} [Sun Jul 17 16:41:51 2022] GET /identity/v3/services/image => generated 86 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:41:51.584495 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11633966-e8e5-4037-ba70-e05b3658e6a3 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:51.759397 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11633966-e8e5-4037-ba70-e05b3658e6a3 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:51.775089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:51.833417 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3168668-86d4-4e95-8d67-193fb0a8ed5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:51.834638 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3168668-86d4-4e95-8d67-193fb0a8ed5e None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:51.835137 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3168668-86d4-4e95-8d67-193fb0a8ed5e None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:51.835724 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3168668-86d4-4e95-8d67-193fb0a8ed5e None admin] PATH_INFO: `/v3/services` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:51.837260 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3168668-86d4-4e95-8d67-193fb0a8ed5e None admin] RBAC: Adding query filter params (name=image) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:51.837827 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3168668-86d4-4e95-8d67-193fb0a8ed5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:51.840330 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3168668-86d4-4e95-8d67-193fb0a8ed5e None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:51.856846 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 516/1024] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:51 2022] GET /identity/v3/services?name=image => generated 122 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:51.861138 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ef45db-3849-4916-a8ca-f3e91864b178 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:52.064373 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ef45db-3849-4916-a8ca-f3e91864b178 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:52.067550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:52.138124 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c19af-f603-4492-a239-30b956db02b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:52.139297 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c19af-f603-4492-a239-30b956db02b6 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:52.139815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c19af-f603-4492-a239-30b956db02b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:52.140322 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c19af-f603-4492-a239-30b956db02b6 None admin] PATH_INFO: `/v3/services` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:52.141743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c19af-f603-4492-a239-30b956db02b6 None admin] RBAC: Adding query filter params (type=image) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:52.142301 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c19af-f603-4492-a239-30b956db02b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:52.145070 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c19af-f603-4492-a239-30b956db02b6 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:52.160887 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 509/1025] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:51 2022] GET /identity/v3/services?type=image => generated 350 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:52.169365 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3168668-86d4-4e95-8d67-193fb0a8ed5e None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:52.250618 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513480d-25a9-4237-905a-2bdc6c8e9a12 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:52.251397 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513480d-25a9-4237-905a-2bdc6c8e9a12 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:52.251564 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513480d-25a9-4237-905a-2bdc6c8e9a12 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:52.253779 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 510/1026] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:41:52.260201 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ffafa9-f4bd-4665-a9f5-03f42cc01b00 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:52.260651 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ffafa9-f4bd-4665-a9f5-03f42cc01b00 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:52.261155 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ffafa9-f4bd-4665-a9f5-03f42cc01b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:52.281962 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-16ffafa9-f4bd-4665-a9f5-03f42cc01b00 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:52.283559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-16ffafa9-f4bd-4665-a9f5-03f42cc01b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:52.378154 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3168668-86d4-4e95-8d67-193fb0a8ed5e None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:52.389041 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 511/1027] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:52 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 17 16:41:52.433593 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ffafa9-f4bd-4665-a9f5-03f42cc01b00 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:52.440429 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d782c-1098-4609-837f-5009de9406a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:52.441835 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d782c-1098-4609-837f-5009de9406a8 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:52.442422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d782c-1098-4609-837f-5009de9406a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:52.442926 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d782c-1098-4609-837f-5009de9406a8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:52.444435 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d782c-1098-4609-837f-5009de9406a8 None admin] RBAC: Adding query filter params (interface=public, service_id=904334c1c4834beb95d40ed750b8e9e6, region_id=RegionOne) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:52.445014 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d782c-1098-4609-837f-5009de9406a8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:52.447429 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d782c-1098-4609-837f-5009de9406a8 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:52.464902 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 517/1028] 10.222.0.40 () {56 vars in 1251 bytes} [Sun Jul 17 16:41:52 2022] GET /identity/v3/endpoints?service_id=904334c1c4834beb95d40ed750b8e9e6&interface=public®ion_id=RegionOne => generated 194 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:52.467100 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d782c-1098-4609-837f-5009de9406a8 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:52.525418 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:52.603461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:52.606770 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d782c-1098-4609-837f-5009de9406a8 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:52.610570 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ffafa9-f4bd-4665-a9f5-03f42cc01b00 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:52.652288 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b978a43-5c1d-40f3-9586-20be1a73e361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:52.653316 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b978a43-5c1d-40f3-9586-20be1a73e361 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:52.653615 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b978a43-5c1d-40f3-9586-20be1a73e361 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:52.653892 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b978a43-5c1d-40f3-9586-20be1a73e361 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:52.655162 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b978a43-5c1d-40f3-9586-20be1a73e361 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:52.656619 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b978a43-5c1d-40f3-9586-20be1a73e361 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:52.658420 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 512/1029] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:52.662166 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d40cef3-f60c-402f-a31f-dc9104615862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:52.662561 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7b7e9-4428-4c5a-a977-0fe14285b7b1 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:52.662939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7b7e9-4428-4c5a-a977-0fe14285b7b1 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:52.663454 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7b7e9-4428-4c5a-a977-0fe14285b7b1 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:52.663706 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40cef3-f60c-402f-a31f-dc9104615862 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:52.664233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40cef3-f60c-402f-a31f-dc9104615862 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:52.664730 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40cef3-f60c-402f-a31f-dc9104615862 None admin] PATH_INFO: `/v3/services` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:52.665292 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 513/1030] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:52.666091 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d40cef3-f60c-402f-a31f-dc9104615862 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:52.666250 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f7b7e9-4428-4c5a-a977-0fe14285b7b1 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:52.669113 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d40cef3-f60c-402f-a31f-dc9104615862 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:52.692827 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 518/1031] 10.222.0.40 () {56 vars in 1087 bytes} [Sun Jul 17 16:41:52 2022] GET /identity/v3/services => generated 1957 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 17 16:41:52.692827 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa4ab2-6f39-4bc8-ad70-2b55e27cdec0 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:52.692827 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa4ab2-6f39-4bc8-ad70-2b55e27cdec0 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:52.692827 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa4ab2-6f39-4bc8-ad70-2b55e27cdec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:52.718184 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-02fa4ab2-6f39-4bc8-ad70-2b55e27cdec0 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:52.720034 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-02fa4ab2-6f39-4bc8-ad70-2b55e27cdec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:52.817194 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 519/1032] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:52 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:41:52.829453 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f7b7e9-4428-4c5a-a977-0fe14285b7b1 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:52.843356 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa4ab2-6f39-4bc8-ad70-2b55e27cdec0 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:52.874576 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:52.880487 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910827fc-d238-489b-95c7-108abd96bcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:52.881232 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910827fc-d238-489b-95c7-108abd96bcd2 None admin] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:52.881501 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910827fc-d238-489b-95c7-108abd96bcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:52.881785 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910827fc-d238-489b-95c7-108abd96bcd2 None admin] PATH_INFO: `/v3/users` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:52.882772 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910827fc-d238-489b-95c7-108abd96bcd2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:52.883968 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910827fc-d238-489b-95c7-108abd96bcd2 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:52.964445 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa4ab2-6f39-4bc8-ad70-2b55e27cdec0 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:53.002994 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 514/1033] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 17 16:41:52 2022] POST /identity/v3/users => generated 285 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:41:53.013546 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c325e8f-4fca-47f9-bafa-694a76d1e857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:53.014602 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c325e8f-4fca-47f9-bafa-694a76d1e857 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:53.015001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c325e8f-4fca-47f9-bafa-694a76d1e857 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:53.015402 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c325e8f-4fca-47f9-bafa-694a76d1e857 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:53.030012 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c325e8f-4fca-47f9-bafa-694a76d1e857 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:41:53.039129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-1c325e8f-4fca-47f9-bafa-694a76d1e857 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 17 16:41:53.039129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:53.039129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:53.039129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:53.039129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:53.039129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:53.039129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:53.039129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:53.039129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:53.039129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:53.039129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:53.039129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:53.039129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:41:53.039129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:41:53.039129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:41:53.039129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:41:53.041545 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:41:53.041545 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:41:53.041545 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:41:53.041545 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:41:53.041545 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:53.041545 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:53.041545 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:53.041545 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:53.041545 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:53.041545 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:53.041545 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:53.041545 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:53.041545 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:53.041545 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:53.041545 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:53.041545 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:53.042784 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:53.042784 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:53.042784 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:53.042784 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:53.042784 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:41:53.042784 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:41:53.042784 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:41:53.042784 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:41:53.042784 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:41:53.042784 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:41:53.042784 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 17 16:41:53.042784 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:53.042784 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 520/1034] 10.222.0.40 () {56 vars in 1120 bytes} [Sun Jul 17 16:41:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:41:53.048828 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910827fc-d238-489b-95c7-108abd96bcd2 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:53.095925 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:53.277424 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910827fc-d238-489b-95c7-108abd96bcd2 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:53.347130 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2132c7a9-3d2c-4733-8d18-3123652698a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:53.348245 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132c7a9-3d2c-4733-8d18-3123652698a5 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:53.348741 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132c7a9-3d2c-4733-8d18-3123652698a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:53.349243 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132c7a9-3d2c-4733-8d18-3123652698a5 None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:53.350676 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2132c7a9-3d2c-4733-8d18-3123652698a5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:53.351237 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2132c7a9-3d2c-4733-8d18-3123652698a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:53.353793 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2132c7a9-3d2c-4733-8d18-3123652698a5 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:53.376829 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 515/1035] 10.222.0.40 () {56 vars in 1115 bytes} [Sun Jul 17 16:41:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:53.381649 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c325e8f-4fca-47f9-bafa-694a76d1e857 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:53.562871 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:53.586469 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c325e8f-4fca-47f9-bafa-694a76d1e857 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:53.658193 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17abffd-36fb-48a0-92e2-13475b2674e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:53.659473 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17abffd-36fb-48a0-92e2-13475b2674e4 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:53.659979 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17abffd-36fb-48a0-92e2-13475b2674e4 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:53.660486 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17abffd-36fb-48a0-92e2-13475b2674e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:53.693084 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17abffd-36fb-48a0-92e2-13475b2674e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:53.695868 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17abffd-36fb-48a0-92e2-13475b2674e4 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:53.731782 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-a17abffd-36fb-48a0-92e2-13475b2674e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:53.731782 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:53.731782 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:53.731782 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:53.731782 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:53.731782 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:53.731782 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:53.731782 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:53.731782 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:53.731782 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:53.731782 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:53.731782 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:53.731782 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:41:53.731782 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:41:53.731782 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:41:53.733713 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:41:53.733713 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:53.733713 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:53.733713 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:53.733713 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:53.733713 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:53.733713 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:53.733713 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:53.733713 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:53.733713 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:53.733713 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:53.733713 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:53.733713 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:53.733713 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:53.733713 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:53.733713 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:53.733713 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:53.735512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:41:53.735512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:41:53.735512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:53.735512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:53.737403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 521/1036] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:53.744000 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2132c7a9-3d2c-4733-8d18-3123652698a5 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:53.904397 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:53.930335 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2132c7a9-3d2c-4733-8d18-3123652698a5 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:54.000798 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a239a-9299-4ff4-a1c4-a6360d343ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:54.001963 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a239a-9299-4ff4-a1c4-a6360d343ca1 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:54.002467 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a239a-9299-4ff4-a1c4-a6360d343ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:54.002945 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a239a-9299-4ff4-a1c4-a6360d343ca1 None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:54.004084 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802a239a-9299-4ff4-a1c4-a6360d343ca1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:54.004505 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802a239a-9299-4ff4-a1c4-a6360d343ca1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:54.006741 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802a239a-9299-4ff4-a1c4-a6360d343ca1 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:54.023454 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 516/1037] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:41:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:54.029780 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17abffd-36fb-48a0-92e2-13475b2674e4 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:54.182613 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17abffd-36fb-48a0-92e2-13475b2674e4 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:54.224691 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a323e915-00cf-4c64-ac2d-5de051b7db11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:54.225521 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323e915-00cf-4c64-ac2d-5de051b7db11 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:54.225805 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323e915-00cf-4c64-ac2d-5de051b7db11 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:54.226086 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a323e915-00cf-4c64-ac2d-5de051b7db11 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:54.234596 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a323e915-00cf-4c64-ac2d-5de051b7db11 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:54.236077 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a323e915-00cf-4c64-ac2d-5de051b7db11 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:54.239406 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:54.255768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-a323e915-00cf-4c64-ac2d-5de051b7db11 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 17 16:41:54.255768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:54.255768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:54.255768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:54.255768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:54.255768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:54.255768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:54.255768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:54.255768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:54.255768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:54.255768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:54.255768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:54.255768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:41:54.255768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:41:54.255768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:41:54.255768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:41:54.257691 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:54.257691 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:54.257691 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:41:54.257691 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:41:54.257691 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:41:54.257691 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:41:54.257691 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:54.257691 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:54.257691 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:54.257691 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:54.257691 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:54.257691 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:54.257691 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:54.257691 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:54.257691 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:54.257691 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:54.257691 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:54.257691 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:54.259776 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:54.259776 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:54.259776 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:41:54.259776 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:41:54.259776 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:41:54.259776 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:41:54.259776 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:41:54.259776 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:41:54.259776 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 17 16:41:54.259776 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:54.259776 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 522/1038] 10.222.0.40 () {56 vars in 1093 bytes} [Sun Jul 17 16:41:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:41:54.272401 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a239a-9299-4ff4-a1c4-a6360d343ca1 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:54.432259 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a239a-9299-4ff4-a1c4-a6360d343ca1 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:54.492771 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05f293-81b6-4764-b0a6-f317dc56e744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:54.493987 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05f293-81b6-4764-b0a6-f317dc56e744 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:54.494481 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05f293-81b6-4764-b0a6-f317dc56e744 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:54.494987 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05f293-81b6-4764-b0a6-f317dc56e744 None admin] PATH_INFO: `/v3/users` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:54.498228 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05f293-81b6-4764-b0a6-f317dc56e744 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:54.498804 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05f293-81b6-4764-b0a6-f317dc56e744 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:54.500939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05f293-81b6-4764-b0a6-f317dc56e744 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:54.529435 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:54.542285 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 517/1039] 10.222.0.40 () {56 vars in 1133 bytes} [Sun Jul 17 16:41:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:54.544673 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a323e915-00cf-4c64-ac2d-5de051b7db11 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:54.707236 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a323e915-00cf-4c64-ac2d-5de051b7db11 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:54.751241 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35370487-e50c-4c36-bc71-1fc6ca17d5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:54.751917 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35370487-e50c-4c36-bc71-1fc6ca17d5a8 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:54.752204 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35370487-e50c-4c36-bc71-1fc6ca17d5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:54.752486 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35370487-e50c-4c36-bc71-1fc6ca17d5a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:54.828630 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:54.831914 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35370487-e50c-4c36-bc71-1fc6ca17d5a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:54.833436 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35370487-e50c-4c36-bc71-1fc6ca17d5a8 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:54.960851 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38881529-fe70-428a-afff-a01b914d07b8 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:54.961171 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38881529-fe70-428a-afff-a01b914d07b8 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:54.961600 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38881529-fe70-428a-afff-a01b914d07b8 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:54.963894 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 518/1040] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:41:54.972363 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e268765-b3c7-430c-8219-d7254a1521fa None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:54.972644 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e268765-b3c7-430c-8219-d7254a1521fa None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:54.973455 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e268765-b3c7-430c-8219-d7254a1521fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:55.000057 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 523/1041] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:41:54 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 17 16:41:55.001412 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-6e268765-b3c7-430c-8219-d7254a1521fa None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:55.003449 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-6e268765-b3c7-430c-8219-d7254a1521fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:55.005733 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35370487-e50c-4c36-bc71-1fc6ca17d5a8 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:55.124121 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 519/1042] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:54 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 17 16:41:55.145081 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:55.150339 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35370487-e50c-4c36-bc71-1fc6ca17d5a8 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:55.154413 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e268765-b3c7-430c-8219-d7254a1521fa None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:55.214803 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f1b19-8c6c-48b7-9459-52a1629f0908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:55.216071 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f1b19-8c6c-48b7-9459-52a1629f0908 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:55.216356 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f1b19-8c6c-48b7-9459-52a1629f0908 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:55.216673 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f1b19-8c6c-48b7-9459-52a1629f0908 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:55.229226 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f1b19-8c6c-48b7-9459-52a1629f0908 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:55.231282 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f1b19-8c6c-48b7-9459-52a1629f0908 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:55.254712 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-4e7f1b19-8c6c-48b7-9459-52a1629f0908 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:55.254712 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:55.254712 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:55.254712 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:55.254712 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:55.254712 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:55.254712 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:55.254712 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:55.254712 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:55.254712 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:55.254712 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:55.254712 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:55.254712 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:41:55.254712 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:41:55.254712 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:41:55.256177 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:41:55.256177 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:55.256177 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:55.256177 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:55.256177 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:55.256177 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:55.256177 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:55.256177 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:55.256177 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:55.256177 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:55.256177 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:55.256177 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:55.256177 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:55.256177 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:55.256177 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:55.256177 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:55.256177 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:55.257427 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:41:55.257427 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:41:55.257427 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:55.257427 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:55.257427 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 524/1043] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:41:55.259450 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681076ea-b31f-40bb-b7c5-b23d1cfb07b5 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:55.259867 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681076ea-b31f-40bb-b7c5-b23d1cfb07b5 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:55.260266 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681076ea-b31f-40bb-b7c5-b23d1cfb07b5 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:55.262599 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 525/1044] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:55.263928 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681076ea-b31f-40bb-b7c5-b23d1cfb07b5 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:55.307372 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e268765-b3c7-430c-8219-d7254a1521fa None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:55.343089 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:55.351112 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77f854-e1da-455f-abfb-d48a8b334cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:55.352254 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77f854-e1da-455f-abfb-d48a8b334cf6 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:55.352730 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77f854-e1da-455f-abfb-d48a8b334cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:55.355480 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77f854-e1da-455f-abfb-d48a8b334cf6 None admin] PATH_INFO: `/v3/services/image` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:55.355480 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab77f854-e1da-455f-abfb-d48a8b334cf6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:55.356655 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab77f854-e1da-455f-abfb-d48a8b334cf6 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:55.377663 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-ab77f854-e1da-455f-abfb-d48a8b334cf6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 17 16:41:55.377663 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:55.377663 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 17 16:41:55.377663 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 17 16:41:55.377663 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 17 16:41:55.377663 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 17 16:41:55.377663 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 17 16:41:55.377663 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:55.377663 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 17 16:41:55.377663 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:55.377663 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 17 16:41:55.377663 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:55.377663 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:55.377663 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:55.377663 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:55.377663 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:55.377663 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:55.379603 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:55.379603 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:55.379603 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:55.379603 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:55.379603 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:55.379603 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:55.379603 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 17 16:41:55.379603 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 17 16:41:55.379603 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 17 16:41:55.379603 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 17 16:41:55.379603 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:55.379603 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:55.379603 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:55.379603 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:55.379603 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:55.379603 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:55.381224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:55.381224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:55.381224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:55.381224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:55.381224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:55.381224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:55.381224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:55.381224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:55.381224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:55.381224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:55.381224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 17 16:41:55.381224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:41:55.381224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 17 16:41:55.381224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:55.381224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 520/1045] 10.222.0.40 () {56 vars in 1105 bytes} [Sun Jul 17 16:41:55 2022] GET /identity/v3/services/image => generated 86 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:41:55.382759 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed38ce1-bdf6-419b-b77a-c5c20fadc0c1 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:55.382759 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed38ce1-bdf6-419b-b77a-c5c20fadc0c1 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:55.382759 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed38ce1-bdf6-419b-b77a-c5c20fadc0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:55.408306 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-fed38ce1-bdf6-419b-b77a-c5c20fadc0c1 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:55.410294 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-fed38ce1-bdf6-419b-b77a-c5c20fadc0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:55.426185 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681076ea-b31f-40bb-b7c5-b23d1cfb07b5 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:55.473512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dfb1e4-a541-4ebb-9aff-f8784b53b607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:55.474437 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dfb1e4-a541-4ebb-9aff-f8784b53b607 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:55.474915 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dfb1e4-a541-4ebb-9aff-f8784b53b607 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:55.475364 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dfb1e4-a541-4ebb-9aff-f8784b53b607 None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:55.476621 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dfb1e4-a541-4ebb-9aff-f8784b53b607 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:55.477165 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dfb1e4-a541-4ebb-9aff-f8784b53b607 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:55.479267 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dfb1e4-a541-4ebb-9aff-f8784b53b607 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:55.497712 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 526/1046] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:41:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:55.498706 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dfb1e4-a541-4ebb-9aff-f8784b53b607 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:55.511869 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 521/1047] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:41:55.514114 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed38ce1-bdf6-419b-b77a-c5c20fadc0c1 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:55.525480 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:55.645415 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dfb1e4-a541-4ebb-9aff-f8784b53b607 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:55.650372 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed38ce1-bdf6-419b-b77a-c5c20fadc0c1 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:55.687190 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:55.696901 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d47d4f6-efa2-411d-9c44-479876311a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:55.697779 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d47d4f6-efa2-411d-9c44-479876311a0a None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:55.698107 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d47d4f6-efa2-411d-9c44-479876311a0a None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:55.698820 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d47d4f6-efa2-411d-9c44-479876311a0a None admin] PATH_INFO: `/v3/services` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:55.700172 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d47d4f6-efa2-411d-9c44-479876311a0a None admin] RBAC: Adding query filter params (name=image) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:55.700593 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d47d4f6-efa2-411d-9c44-479876311a0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:55.702235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d47d4f6-efa2-411d-9c44-479876311a0a None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:55.702831 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd43b41-ec8a-4424-8b56-1d143917bcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:55.703978 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd43b41-ec8a-4424-8b56-1d143917bcaa None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:55.704261 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd43b41-ec8a-4424-8b56-1d143917bcaa None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:55.704627 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd43b41-ec8a-4424-8b56-1d143917bcaa None admin] PATH_INFO: `/v3/projects/service` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:55.709201 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 527/1048] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:55 2022] GET /identity/v3/services?name=image => generated 122 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:55.710279 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d47d4f6-efa2-411d-9c44-479876311a0a None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:55.717652 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd43b41-ec8a-4424-8b56-1d143917bcaa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:55.719989 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd43b41-ec8a-4424-8b56-1d143917bcaa None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:55.738208 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-bfd43b41-ec8a-4424-8b56-1d143917bcaa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:41:55.738208 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:55.738208 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:55.738208 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:55.738208 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:55.738208 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:55.738208 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:55.738208 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:55.738208 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:55.738208 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:55.738208 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:55.738208 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:55.738208 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:41:55.738208 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:41:55.738208 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:41:55.740100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:41:55.740100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:55.740100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:55.740100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:55.740100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:55.740100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:55.740100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:55.740100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:55.740100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:55.740100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:55.740100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:55.740100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:55.740100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:55.740100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:55.740100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:55.740100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:55.740100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:55.741962 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:41:55.741962 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:41:55.741962 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:41:55.741962 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:41:55.741962 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:41:55.741962 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:41:55.741962 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:41:55.741962 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:55.741962 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 522/1049] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:41:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:55.741962 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd43b41-ec8a-4424-8b56-1d143917bcaa None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:55.805283 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:55.844084 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d47d4f6-efa2-411d-9c44-479876311a0a None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:55.886605 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd43b41-ec8a-4424-8b56-1d143917bcaa None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:55.895127 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5d2bb-5c4f-46df-a9ec-95aa82f33998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:55.895989 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5d2bb-5c4f-46df-a9ec-95aa82f33998 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:55.896394 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5d2bb-5c4f-46df-a9ec-95aa82f33998 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:55.896741 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5d2bb-5c4f-46df-a9ec-95aa82f33998 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:55.910100 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a5d2bb-5c4f-46df-a9ec-95aa82f33998 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:41:55.918465 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-c6a5d2bb-5c4f-46df-a9ec-95aa82f33998 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:41:55.918465 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:55.918465 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:55.918465 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:55.918465 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:55.918465 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:55.918465 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:55.918465 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:55.918465 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:55.918465 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:55.918465 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:55.918465 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:55.918465 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:41:55.918465 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:41:55.918465 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:41:55.918465 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:41:55.920319 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:41:55.920319 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:41:55.920319 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:41:55.920319 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:41:55.920319 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:55.920319 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:55.920319 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:55.920319 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:55.920319 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:55.920319 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:55.920319 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:55.920319 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:55.920319 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:55.920319 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:55.920319 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:55.920319 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:55.922020 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:55.922020 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:55.922020 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:55.922020 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:55.922020 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:41:55.922020 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:41:55.922020 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:41:55.922020 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:41:55.922020 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:41:55.922020 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:41:55.922020 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:41:55.922020 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:55.922020 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 528/1050] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:41:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:55.922020 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5d2bb-5c4f-46df-a9ec-95aa82f33998 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:55.933332 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ff309-1955-4690-bf79-73b6f87a61cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:55.934482 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ff309-1955-4690-bf79-73b6f87a61cb None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:55.934945 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ff309-1955-4690-bf79-73b6f87a61cb None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:55.935396 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8ff309-1955-4690-bf79-73b6f87a61cb None admin] PATH_INFO: `/v3/services` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:55.936613 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ff309-1955-4690-bf79-73b6f87a61cb None admin] RBAC: Adding query filter params (type=image) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:55.937140 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ff309-1955-4690-bf79-73b6f87a61cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:55.939272 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8ff309-1955-4690-bf79-73b6f87a61cb None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:55.948474 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 523/1051] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:55 2022] GET /identity/v3/services?type=image => generated 350 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:55.950198 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ff309-1955-4690-bf79-73b6f87a61cb None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:56.020558 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:56.074876 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5d2bb-5c4f-46df-a9ec-95aa82f33998 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:56.099649 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8ff309-1955-4690-bf79-73b6f87a61cb None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:56.126145 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a383f6-83d5-4622-9df9-341b0b8def19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:56.126996 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a383f6-83d5-4622-9df9-341b0b8def19 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:56.127340 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a383f6-83d5-4622-9df9-341b0b8def19 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:56.127676 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a383f6-83d5-4622-9df9-341b0b8def19 None admin] PATH_INFO: `/v3/projects` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:56.128548 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a383f6-83d5-4622-9df9-341b0b8def19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:56.129043 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a383f6-83d5-4622-9df9-341b0b8def19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:56.130891 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a383f6-83d5-4622-9df9-341b0b8def19 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:56.134425 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:56.148509 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 529/1052] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:41:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:56.149449 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4d826-b531-41fc-bbda-f2bd3d1c4344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:56.149760 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a383f6-83d5-4622-9df9-341b0b8def19 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:56.150592 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4d826-b531-41fc-bbda-f2bd3d1c4344 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:56.151082 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4d826-b531-41fc-bbda-f2bd3d1c4344 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:56.151567 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4d826-b531-41fc-bbda-f2bd3d1c4344 None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:56.154283 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4d826-b531-41fc-bbda-f2bd3d1c4344 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:56.154824 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4d826-b531-41fc-bbda-f2bd3d1c4344 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:56.157441 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4d826-b531-41fc-bbda-f2bd3d1c4344 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:56.176683 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 524/1053] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:41:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:56.176683 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4d826-b531-41fc-bbda-f2bd3d1c4344 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:56.293102 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a383f6-83d5-4622-9df9-341b0b8def19 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:56.305299 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4d826-b531-41fc-bbda-f2bd3d1c4344 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:56.324206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:56.348133 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b36f3-3c01-4c11-9cf1-a0b26c76f605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:56.349350 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b36f3-3c01-4c11-9cf1-a0b26c76f605 None admin] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:56.349765 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b36f3-3c01-4c11-9cf1-a0b26c76f605 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:56.350220 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b36f3-3c01-4c11-9cf1-a0b26c76f605 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:56.351496 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b36f3-3c01-4c11-9cf1-a0b26c76f605 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:56.353873 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b36f3-3c01-4c11-9cf1-a0b26c76f605 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:56.364468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c035ce2-6e37-4f55-b3f0-2e39e28e6786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:56.365573 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c035ce2-6e37-4f55-b3f0-2e39e28e6786 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:56.366388 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c035ce2-6e37-4f55-b3f0-2e39e28e6786 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:56.367172 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c035ce2-6e37-4f55-b3f0-2e39e28e6786 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:56.368401 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c035ce2-6e37-4f55-b3f0-2e39e28e6786 None admin] RBAC: Adding query filter params (role.id=68d3a95a9fc34fd1a28a9861d22ee828, scope.project.id=927057f9a5af4833aeac9992203a223c, user.id=03244087f1a34bddb11ea46aba11a148) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:56.368852 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c035ce2-6e37-4f55-b3f0-2e39e28e6786 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:56.370754 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c035ce2-6e37-4f55-b3f0-2e39e28e6786 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:56.377800 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 525/1054] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:41:56 2022] GET /identity/v3/role_assignments?user.id=03244087f1a34bddb11ea46aba11a148&scope.project.id=927057f9a5af4833aeac9992203a223c&role.id=68d3a95a9fc34fd1a28a9861d22ee828 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:56.378829 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c035ce2-6e37-4f55-b3f0-2e39e28e6786 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:56.450049 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 530/1055] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 17 16:41:56 2022] POST /identity/v3/endpoints => generated 327 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:41:56.516441 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c035ce2-6e37-4f55-b3f0-2e39e28e6786 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:56.572103 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1981c-e70b-466b-96ee-31adb0a9ea1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:56.572706 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1981c-e70b-466b-96ee-31adb0a9ea1d None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:56.574297 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1981c-e70b-466b-96ee-31adb0a9ea1d None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:56.574297 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1981c-e70b-466b-96ee-31adb0a9ea1d None admin] PATH_INFO: `/v3/users/738cc6e7b90343cdbdff27d64e3f98ea` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:56.594716 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b1981c-e70b-466b-96ee-31adb0a9ea1d None admin] RBAC: Authorizing `identity:get_user(user_id=738cc6e7b90343cdbdff27d64e3f98ea)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:56.596201 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b1981c-e70b-466b-96ee-31adb0a9ea1d None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:56.598258 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 526/1056] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:41:56 2022] GET /identity/v3/users/738cc6e7b90343cdbdff27d64e3f98ea => generated 285 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:56.604043 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b36f3-3c01-4c11-9cf1-a0b26c76f605 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:56.672748 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:56.753811 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b36f3-3c01-4c11-9cf1-a0b26c76f605 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:56.802263 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450b8a4-a08c-42b4-979e-4623849b97b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:56.803106 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450b8a4-a08c-42b4-979e-4623849b97b4 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:56.803524 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450b8a4-a08c-42b4-979e-4623849b97b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:56.804039 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450b8a4-a08c-42b4-979e-4623849b97b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:56.890702 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450b8a4-a08c-42b4-979e-4623849b97b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:56.892096 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450b8a4-a08c-42b4-979e-4623849b97b4 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:56.987614 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:57.084827 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 531/1057] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:41:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 17 16:41:57.088481 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1981c-e70b-466b-96ee-31adb0a9ea1d None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:57.146747 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:57.297302 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1981c-e70b-466b-96ee-31adb0a9ea1d None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:57.360132 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd6e18-5335-4aed-9372-b93df6f71058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:57.361688 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd6e18-5335-4aed-9372-b93df6f71058 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:57.361984 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd6e18-5335-4aed-9372-b93df6f71058 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:57.362447 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd6e18-5335-4aed-9372-b93df6f71058 None admin] PATH_INFO: `/v3/projects/1efc8a7f7d1d47b9955fad77a83260b7` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:57.364595 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd6e18-5335-4aed-9372-b93df6f71058 None admin] RBAC: Authorizing `identity:get_project(project_id=1efc8a7f7d1d47b9955fad77a83260b7)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:57.367394 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd6e18-5335-4aed-9372-b93df6f71058 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:57.376223 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 527/1058] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:41:57 2022] GET /identity/v3/projects/1efc8a7f7d1d47b9955fad77a83260b7 => generated 311 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:57.381846 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450b8a4-a08c-42b4-979e-4623849b97b4 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:57.557271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450b8a4-a08c-42b4-979e-4623849b97b4 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:57.600337 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af63095c-a924-4705-a11a-5e3959c8a0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:57.601049 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63095c-a924-4705-a11a-5e3959c8a0a9 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:57.601334 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63095c-a924-4705-a11a-5e3959c8a0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:57.612888 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63095c-a924-4705-a11a-5e3959c8a0a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:57.612888 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af63095c-a924-4705-a11a-5e3959c8a0a9 None admin] RBAC: Adding query filter params (role.id=1025051975f64f689effa58dc4259e36, scope.project.id=1efc8a7f7d1d47b9955fad77a83260b7, user.id=738cc6e7b90343cdbdff27d64e3f98ea) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:57.612888 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af63095c-a924-4705-a11a-5e3959c8a0a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:57.612888 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af63095c-a924-4705-a11a-5e3959c8a0a9 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:57.612888 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 532/1059] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:41:57 2022] GET /identity/v3/role_assignments?user.id=738cc6e7b90343cdbdff27d64e3f98ea&scope.project.id=1efc8a7f7d1d47b9955fad77a83260b7&role.id=1025051975f64f689effa58dc4259e36 => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:41:58.524587 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a5998-c96c-4e00-8b71-15dd35dadc21 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:58.525492 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a5998-c96c-4e00-8b71-15dd35dadc21 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:58.525492 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a5998-c96c-4e00-8b71-15dd35dadc21 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:58.527909 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 528/1060] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:41:58.534555 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca35bad-4b88-43cc-b9d4-e1df5fd57bc6 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:58.535388 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca35bad-4b88-43cc-b9d4-e1df5fd57bc6 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:58.535545 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca35bad-4b88-43cc-b9d4-e1df5fd57bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:58.555400 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-eca35bad-4b88-43cc-b9d4-e1df5fd57bc6 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:58.556954 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-eca35bad-4b88-43cc-b9d4-e1df5fd57bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:58.571407 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:58.648857 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 533/1061] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 17 16:41:58.673403 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a5998-c96c-4e00-8b71-15dd35dadc21 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:58.745964 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe15b75-3cf0-48b7-a1b1-e74e96a028af None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:58.746990 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe15b75-3cf0-48b7-a1b1-e74e96a028af None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:58.747496 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe15b75-3cf0-48b7-a1b1-e74e96a028af None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:58.750144 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 534/1062] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:58.757017 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e4349-1d5a-496e-898e-2d878eed83ec None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:58.757321 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e4349-1d5a-496e-898e-2d878eed83ec None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:58.757624 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e4349-1d5a-496e-898e-2d878eed83ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:58.783201 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-758e4349-1d5a-496e-898e-2d878eed83ec None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:58.784357 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-758e4349-1d5a-496e-898e-2d878eed83ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:41:58.817025 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:58.830055 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a5998-c96c-4e00-8b71-15dd35dadc21 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:58.881009 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 535/1063] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 17 16:41:58.892852 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b09e6c0-85f6-463c-961d-fca1c7fd6fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:58.892852 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09e6c0-85f6-463c-961d-fca1c7fd6fa9 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:58.892852 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09e6c0-85f6-463c-961d-fca1c7fd6fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:58.893382 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09e6c0-85f6-463c-961d-fca1c7fd6fa9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:58.903124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e4349-1d5a-496e-898e-2d878eed83ec None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:58.913429 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b09e6c0-85f6-463c-961d-fca1c7fd6fa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:41:58.930052 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-6b09e6c0-85f6-463c-961d-fca1c7fd6fa9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 17 16:41:58.930052 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:58.930052 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:58.930052 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:58.930052 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:58.930052 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:58.930052 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:58.930052 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:58.930052 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:58.930052 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:58.930052 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:58.930052 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:58.930052 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:41:58.930052 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:41:58.930052 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:41:58.930052 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:41:58.931180 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:41:58.931180 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:41:58.931180 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:41:58.931180 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:41:58.931180 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:58.931180 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:58.931180 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:58.931180 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:58.931180 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:58.931180 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:58.931180 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:58.931180 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:58.931180 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:58.931180 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:58.931180 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:58.931180 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:58.932077 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:58.932077 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:58.932077 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:58.932077 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:58.932077 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:41:58.932077 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:41:58.932077 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:41:58.932077 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:41:58.932077 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:41:58.932077 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:41:58.932077 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 17 16:41:58.932077 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:58.932731 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 529/1064] 10.222.0.40 () {56 vars in 1120 bytes} [Sun Jul 17 16:41:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:41:58.940707 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b09e6c0-85f6-463c-961d-fca1c7fd6fa9 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:59.016849 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:59.048762 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e4349-1d5a-496e-898e-2d878eed83ec None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:59.110541 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef838b6-0a1c-4978-acd3-2f45d2a6e255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:59.111353 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef838b6-0a1c-4978-acd3-2f45d2a6e255 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:59.111776 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef838b6-0a1c-4978-acd3-2f45d2a6e255 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:59.112179 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef838b6-0a1c-4978-acd3-2f45d2a6e255 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:59.125995 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b09e6c0-85f6-463c-961d-fca1c7fd6fa9 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:59.198166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272ddb3-ddb8-4668-a916-32a997203207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:59.200015 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272ddb3-ddb8-4668-a916-32a997203207 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:59.200241 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272ddb3-ddb8-4668-a916-32a997203207 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:59.200734 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272ddb3-ddb8-4668-a916-32a997203207 None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:59.202323 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9272ddb3-ddb8-4668-a916-32a997203207 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:59.202848 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9272ddb3-ddb8-4668-a916-32a997203207 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:59.205469 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9272ddb3-ddb8-4668-a916-32a997203207 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:59.219763 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef838b6-0a1c-4978-acd3-2f45d2a6e255 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:59.221595 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef838b6-0a1c-4978-acd3-2f45d2a6e255 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:59.222922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 530/1065] 10.222.0.40 () {56 vars in 1115 bytes} [Sun Jul 17 16:41:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:41:59.230458 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272ddb3-ddb8-4668-a916-32a997203207 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:59.267370 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:59.355115 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:59.398435 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 536/1066] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:41:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 17 16:41:59.405340 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef838b6-0a1c-4978-acd3-2f45d2a6e255 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:59.438931 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272ddb3-ddb8-4668-a916-32a997203207 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:59.521602 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4840dd4b-d573-4a2b-95d9-3cbacc286175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:59.522616 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840dd4b-d573-4a2b-95d9-3cbacc286175 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:59.523072 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840dd4b-d573-4a2b-95d9-3cbacc286175 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:59.523774 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840dd4b-d573-4a2b-95d9-3cbacc286175 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:59.544123 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4840dd4b-d573-4a2b-95d9-3cbacc286175 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:59.546706 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4840dd4b-d573-4a2b-95d9-3cbacc286175 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:59.551906 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef838b6-0a1c-4978-acd3-2f45d2a6e255 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:59.575977 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-4840dd4b-d573-4a2b-95d9-3cbacc286175 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:59.575977 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:59.575977 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:59.575977 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:59.575977 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:59.575977 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:59.575977 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:59.575977 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:59.575977 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:59.575977 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:59.575977 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:59.575977 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:59.575977 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:41:59.575977 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:41:59.575977 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:41:59.577820 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:41:59.577820 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:59.577820 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:59.577820 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:59.577820 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:59.577820 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:59.577820 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:59.577820 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:59.577820 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:59.577820 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:59.577820 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:59.577820 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:59.577820 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:59.577820 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:59.577820 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:59.577820 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:59.577820 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:59.579517 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:41:59.579517 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:41:59.579517 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:59.579517 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:41:59.579517 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 531/1067] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:41:59.586179 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4840dd4b-d573-4a2b-95d9-3cbacc286175 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:59.625682 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57de069-53c1-4c92-bef2-764c5f4bdab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:59.626811 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57de069-53c1-4c92-bef2-764c5f4bdab9 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:59.627245 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57de069-53c1-4c92-bef2-764c5f4bdab9 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:59.627640 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57de069-53c1-4c92-bef2-764c5f4bdab9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:59.648208 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57de069-53c1-4c92-bef2-764c5f4bdab9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:59.650883 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57de069-53c1-4c92-bef2-764c5f4bdab9 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:59.684084 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-d57de069-53c1-4c92-bef2-764c5f4bdab9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:59.684084 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:41:59.684084 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:41:59.684084 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:41:59.684084 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:41:59.684084 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:41:59.684084 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:41:59.684084 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:41:59.684084 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:41:59.684084 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:41:59.684084 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:41:59.684084 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:41:59.684084 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:41:59.684084 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:41:59.684084 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:41:59.686284 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:41:59.686284 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:41:59.686284 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:41:59.686284 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:41:59.686284 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:41:59.686284 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:41:59.686284 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:41:59.686284 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:41:59.686284 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:41:59.686284 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:41:59.686284 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:41:59.686284 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:41:59.686284 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:41:59.686284 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:41:59.686284 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:41:59.686284 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:41:59.686284 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:41:59.688074 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:41:59.688074 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:41:59.688074 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:41:59.688074 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:41:59.688074 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 537/1068] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:41:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:41:59.694238 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57de069-53c1-4c92-bef2-764c5f4bdab9 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:59.721565 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:59.745288 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:41:59.778986 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4840dd4b-d573-4a2b-95d9-3cbacc286175 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:59.839227 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153602c-4916-4b58-a771-dff8afbe71c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:59.841448 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153602c-4916-4b58-a771-dff8afbe71c7 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:59.841787 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153602c-4916-4b58-a771-dff8afbe71c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:59.842142 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153602c-4916-4b58-a771-dff8afbe71c7 None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:59.843159 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153602c-4916-4b58-a771-dff8afbe71c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:59.843673 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153602c-4916-4b58-a771-dff8afbe71c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:59.846113 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153602c-4916-4b58-a771-dff8afbe71c7 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:59.868727 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 532/1069] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:41:59.875835 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57de069-53c1-4c92-bef2-764c5f4bdab9 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:41:59.878123 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153602c-4916-4b58-a771-dff8afbe71c7 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:41:59.934656 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcf0abc-98bc-4bb6-b84d-283a8644ae35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:41:59.935495 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf0abc-98bc-4bb6-b84d-283a8644ae35 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:59.935870 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf0abc-98bc-4bb6-b84d-283a8644ae35 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:59.936255 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf0abc-98bc-4bb6-b84d-283a8644ae35 None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:59.937188 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcf0abc-98bc-4bb6-b84d-283a8644ae35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:41:59.937591 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcf0abc-98bc-4bb6-b84d-283a8644ae35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:41:59.939164 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcf0abc-98bc-4bb6-b84d-283a8644ae35 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:41:59.954412 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 538/1070] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:41:59.956024 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914df805-01db-42c5-9002-f6c7e145c4ed None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:59.956429 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914df805-01db-42c5-9002-f6c7e145c4ed None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:59.956833 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914df805-01db-42c5-9002-f6c7e145c4ed None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:59.958608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 539/1071] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:41:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:41:59.965531 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734dc4a9-6e84-4f56-9c82-31d18afd4e3b None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:41:59.965953 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734dc4a9-6e84-4f56-9c82-31d18afd4e3b None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:41:59.966347 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734dc4a9-6e84-4f56-9c82-31d18afd4e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:41:59.987126 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-734dc4a9-6e84-4f56-9c82-31d18afd4e3b None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:41:59.988893 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-734dc4a9-6e84-4f56-9c82-31d18afd4e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:42:00.077011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153602c-4916-4b58-a771-dff8afbe71c7 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:00.092879 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 540/1072] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:41:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 17 16:42:00.113828 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734dc4a9-6e84-4f56-9c82-31d18afd4e3b None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:00.141777 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:00.144744 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:00.154911 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b173f3e-76c2-45e0-b4dc-5441f7e6a645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:00.156247 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b173f3e-76c2-45e0-b4dc-5441f7e6a645 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:00.156735 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b173f3e-76c2-45e0-b4dc-5441f7e6a645 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:00.163389 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b173f3e-76c2-45e0-b4dc-5441f7e6a645 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:00.176816 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b173f3e-76c2-45e0-b4dc-5441f7e6a645 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:00.176816 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b173f3e-76c2-45e0-b4dc-5441f7e6a645 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:00.210108 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-8b173f3e-76c2-45e0-b4dc-5441f7e6a645 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 17 16:42:00.210108 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:42:00.210108 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:42:00.210108 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:42:00.210108 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:42:00.210108 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:42:00.210108 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:42:00.210108 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:42:00.210108 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:42:00.210108 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:42:00.210108 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:42:00.210108 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:42:00.210108 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:42:00.210108 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:42:00.210108 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:42:00.210108 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:42:00.211815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:42:00.211815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:42:00.211815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:42:00.211815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:42:00.211815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:42:00.211815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:42:00.211815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:42:00.211815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:42:00.211815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:42:00.211815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:42:00.211815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:42:00.211815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:42:00.211815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:42:00.211815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:42:00.211815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:42:00.211815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:42:00.211815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:42:00.211815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:42:00.213670 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:42:00.213670 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:42:00.213670 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:42:00.213670 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:42:00.213670 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:42:00.213670 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:42:00.213670 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:42:00.213670 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:42:00.213670 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 17 16:42:00.213670 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:42:00.214799 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 533/1073] 10.222.0.40 () {56 vars in 1093 bytes} [Sun Jul 17 16:41:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:42:00.222123 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b173f3e-76c2-45e0-b4dc-5441f7e6a645 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:00.283072 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734dc4a9-6e84-4f56-9c82-31d18afd4e3b None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:00.332549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693094b-bc00-40d4-861c-e0b6c84325d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:00.333394 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693094b-bc00-40d4-861c-e0b6c84325d5 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:00.333679 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693094b-bc00-40d4-861c-e0b6c84325d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:00.333953 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693094b-bc00-40d4-861c-e0b6c84325d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:00.346280 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f693094b-bc00-40d4-861c-e0b6c84325d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:42:00.354010 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-f693094b-bc00-40d4-861c-e0b6c84325d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:42:00.354010 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:42:00.354010 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:42:00.354010 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:42:00.354010 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:42:00.354010 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:42:00.354010 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:42:00.354010 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:42:00.354010 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:42:00.354010 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:42:00.354010 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:42:00.354010 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:42:00.354010 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:42:00.354010 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:42:00.354010 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:42:00.354010 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:42:00.356437 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:42:00.356437 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:42:00.356437 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:42:00.356437 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:42:00.356437 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:42:00.356437 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:42:00.356437 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:42:00.356437 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:42:00.356437 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:42:00.356437 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:42:00.356437 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:42:00.356437 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:42:00.356437 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:42:00.356437 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:42:00.356437 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:42:00.356437 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:42:00.358119 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:42:00.358119 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:42:00.358119 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:42:00.358119 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:42:00.358119 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:42:00.358119 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:42:00.358119 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:42:00.358119 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:42:00.358119 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:42:00.358119 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:42:00.358119 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:42:00.358119 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:42:00.358119 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 541/1074] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:42:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:42:00.365057 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693094b-bc00-40d4-861c-e0b6c84325d5 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:00.390823 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b173f3e-76c2-45e0-b4dc-5441f7e6a645 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:00.437386 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2912cc-70a1-4447-97e0-351a804030e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:00.438039 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2912cc-70a1-4447-97e0-351a804030e2 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:00.438369 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2912cc-70a1-4447-97e0-351a804030e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:00.438696 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2912cc-70a1-4447-97e0-351a804030e2 None admin] PATH_INFO: `/v3/users` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:00.439887 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2912cc-70a1-4447-97e0-351a804030e2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:42:00.440253 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2912cc-70a1-4447-97e0-351a804030e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:00.442198 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2912cc-70a1-4447-97e0-351a804030e2 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:00.464773 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 534/1075] 10.222.0.40 () {56 vars in 1133 bytes} [Sun Jul 17 16:42:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:00.474083 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:00.474341 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2912cc-70a1-4447-97e0-351a804030e2 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:00.504388 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693094b-bc00-40d4-861c-e0b6c84325d5 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:00.537229 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:00.547158 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6257b605-d708-4ad2-af97-da4716ae365d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:00.547857 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6257b605-d708-4ad2-af97-da4716ae365d None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:00.548143 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6257b605-d708-4ad2-af97-da4716ae365d None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:00.548419 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6257b605-d708-4ad2-af97-da4716ae365d None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:00.549233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6257b605-d708-4ad2-af97-da4716ae365d None admin] RBAC: Adding query filter params (name=admin) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:42:00.549544 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6257b605-d708-4ad2-af97-da4716ae365d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:00.550960 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6257b605-d708-4ad2-af97-da4716ae365d None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:00.562018 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 542/1076] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:42:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:42:00.569062 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6257b605-d708-4ad2-af97-da4716ae365d None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:00.609336 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2912cc-70a1-4447-97e0-351a804030e2 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:00.649804 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05b84c-b8b5-492b-b430-9591c2db84aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:00.651033 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05b84c-b8b5-492b-b430-9591c2db84aa None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:00.651505 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05b84c-b8b5-492b-b430-9591c2db84aa None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:00.651973 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c05b84c-b8b5-492b-b430-9591c2db84aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:00.668274 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c05b84c-b8b5-492b-b430-9591c2db84aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:00.669790 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c05b84c-b8b5-492b-b430-9591c2db84aa None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:00.688549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-5c05b84c-b8b5-492b-b430-9591c2db84aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:42:00.688549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:42:00.688549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:42:00.688549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:42:00.688549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:42:00.688549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:42:00.688549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:42:00.688549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:42:00.688549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:42:00.688549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:42:00.688549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:42:00.688549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:42:00.688549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:42:00.688549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:42:00.688549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:42:00.698011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:42:00.698011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:42:00.698011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:42:00.698011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:42:00.698011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:42:00.698011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:42:00.698011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:42:00.698011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:42:00.698011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:42:00.698011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:42:00.698011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:42:00.698011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:42:00.698011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:42:00.698011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:42:00.698011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:42:00.698011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:42:00.698011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:42:00.699545 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:42:00.699545 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:42:00.699545 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:42:00.699545 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:42:00.699545 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 535/1077] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:42:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:42:00.699945 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05b84c-b8b5-492b-b430-9591c2db84aa None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:00.713706 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6257b605-d708-4ad2-af97-da4716ae365d None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:00.762798 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc92ad-a6db-49e5-a1e5-60ca372f59e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:00.763922 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc92ad-a6db-49e5-a1e5-60ca372f59e9 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:00.764389 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc92ad-a6db-49e5-a1e5-60ca372f59e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:00.764766 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcc92ad-a6db-49e5-a1e5-60ca372f59e9 None admin] PATH_INFO: `/v3/users/738cc6e7b90343cdbdff27d64e3f98ea` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:00.766340 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcc92ad-a6db-49e5-a1e5-60ca372f59e9 None admin] RBAC: Authorizing `identity:get_user(user_id=738cc6e7b90343cdbdff27d64e3f98ea)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:00.768072 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcc92ad-a6db-49e5-a1e5-60ca372f59e9 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:00.770191 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 543/1078] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:42:00 2022] GET /identity/v3/users/738cc6e7b90343cdbdff27d64e3f98ea => generated 285 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:42:00.776222 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc92ad-a6db-49e5-a1e5-60ca372f59e9 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:00.790275 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:00.830936 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:00.834689 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c05b84c-b8b5-492b-b430-9591c2db84aa None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:00.878674 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52bf76b-dad8-4906-9fb5-88f5b9ed29d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:00.879346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52bf76b-dad8-4906-9fb5-88f5b9ed29d1 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:00.879633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52bf76b-dad8-4906-9fb5-88f5b9ed29d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:00.879914 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52bf76b-dad8-4906-9fb5-88f5b9ed29d1 None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:00.880652 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52bf76b-dad8-4906-9fb5-88f5b9ed29d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:42:00.880993 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52bf76b-dad8-4906-9fb5-88f5b9ed29d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:00.882388 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52bf76b-dad8-4906-9fb5-88f5b9ed29d1 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:00.895464 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 536/1079] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:42:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:42:00.901688 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52bf76b-dad8-4906-9fb5-88f5b9ed29d1 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:00.908384 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcc92ad-a6db-49e5-a1e5-60ca372f59e9 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:00.954455 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0cb53-3ab5-4892-b9a6-5248b7491212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:00.955514 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0cb53-3ab5-4892-b9a6-5248b7491212 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:00.955969 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0cb53-3ab5-4892-b9a6-5248b7491212 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:00.956407 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0cb53-3ab5-4892-b9a6-5248b7491212 None admin] PATH_INFO: `/v3/projects/1efc8a7f7d1d47b9955fad77a83260b7` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:00.958387 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d0cb53-3ab5-4892-b9a6-5248b7491212 None admin] RBAC: Authorizing `identity:get_project(project_id=1efc8a7f7d1d47b9955fad77a83260b7)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:00.960710 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d0cb53-3ab5-4892-b9a6-5248b7491212 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:00.967873 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 544/1080] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:42:00 2022] GET /identity/v3/projects/1efc8a7f7d1d47b9955fad77a83260b7 => generated 311 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:42:00.975144 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0cb53-3ab5-4892-b9a6-5248b7491212 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:01.035424 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52bf76b-dad8-4906-9fb5-88f5b9ed29d1 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:01.092457 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013095d1-89b8-4a44-8d62-3ce965d4d4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:01.093114 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:01.094316 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013095d1-89b8-4a44-8d62-3ce965d4d4e3 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:01.094785 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013095d1-89b8-4a44-8d62-3ce965d4d4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:01.095276 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013095d1-89b8-4a44-8d62-3ce965d4d4e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:01.113379 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013095d1-89b8-4a44-8d62-3ce965d4d4e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:01.114983 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013095d1-89b8-4a44-8d62-3ce965d4d4e3 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:01.134036 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0cb53-3ab5-4892-b9a6-5248b7491212 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:01.137461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-013095d1-89b8-4a44-8d62-3ce965d4d4e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:42:01.137461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:42:01.137461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:42:01.137461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:42:01.137461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:42:01.137461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:42:01.137461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:42:01.137461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:42:01.137461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:42:01.137461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:42:01.137461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:42:01.137461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:42:01.137461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:42:01.137461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:42:01.137461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:42:01.139268 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:42:01.139268 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:42:01.139268 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:42:01.139268 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:42:01.139268 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:42:01.139268 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:42:01.139268 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:42:01.139268 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:42:01.139268 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:42:01.139268 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:42:01.139268 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:42:01.139268 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:42:01.139268 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:42:01.139268 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:42:01.139268 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:42:01.139268 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:42:01.139268 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:42:01.141468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:42:01.141468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:42:01.141468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:42:01.141468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:42:01.141468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:42:01.141468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:42:01.141468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:42:01.141468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:42:01.141468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 537/1081] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:42:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:42:01.148457 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013095d1-89b8-4a44-8d62-3ce965d4d4e3 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:01.185566 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:01.197854 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8e200-3188-4ef0-bd94-0218b3066a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:01.200106 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8e200-3188-4ef0-bd94-0218b3066a9c None admin] REQUEST_METHOD: `PUT` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:01.200622 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8e200-3188-4ef0-bd94-0218b3066a9c None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:01.203349 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8e200-3188-4ef0-bd94-0218b3066a9c None admin] PATH_INFO: `/v3/projects/1efc8a7f7d1d47b9955fad77a83260b7/users/738cc6e7b90343cdbdff27d64e3f98ea/roles/1025051975f64f689effa58dc4259e36` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:01.205078 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa8e200-3188-4ef0-bd94-0218b3066a9c None admin] RBAC: Authorizing `identity:create_grant(project_id=1efc8a7f7d1d47b9955fad77a83260b7, user_id=738cc6e7b90343cdbdff27d64e3f98ea, role_id=1025051975f64f689effa58dc4259e36)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:01.207897 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa8e200-3188-4ef0-bd94-0218b3066a9c None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:01.294768 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 545/1082] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 17 16:42:00 2022] PUT /identity/v3/projects/1efc8a7f7d1d47b9955fad77a83260b7/users/738cc6e7b90343cdbdff27d64e3f98ea/roles/1025051975f64f689effa58dc4259e36 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 17 16:42:01.309741 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013095d1-89b8-4a44-8d62-3ce965d4d4e3 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:01.367465 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae58cd-fa38-4c91-8acd-417cdb1927cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:01.368169 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae58cd-fa38-4c91-8acd-417cdb1927cb None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:01.368457 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae58cd-fa38-4c91-8acd-417cdb1927cb None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:01.368738 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae58cd-fa38-4c91-8acd-417cdb1927cb None admin] PATH_INFO: `/v3/projects` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:01.369595 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ae58cd-fa38-4c91-8acd-417cdb1927cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:42:01.371772 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ae58cd-fa38-4c91-8acd-417cdb1927cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:01.374751 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ae58cd-fa38-4c91-8acd-417cdb1927cb None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:01.392902 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 538/1083] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:42:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:42:01.403833 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8e200-3188-4ef0-bd94-0218b3066a9c None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:01.555742 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8e200-3188-4ef0-bd94-0218b3066a9c None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:01.618100 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649cc934-4c35-4785-9606-f189567ca368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:01.619685 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649cc934-4c35-4785-9606-f189567ca368 None admin] REQUEST_METHOD: `PUT` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:01.620178 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649cc934-4c35-4785-9606-f189567ca368 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:01.620681 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649cc934-4c35-4785-9606-f189567ca368 None admin] PATH_INFO: `/v3/projects/927057f9a5af4833aeac9992203a223c/users/03244087f1a34bddb11ea46aba11a148/roles/68d3a95a9fc34fd1a28a9861d22ee828` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:01.624816 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649cc934-4c35-4785-9606-f189567ca368 None admin] RBAC: Authorizing `identity:create_grant(project_id=927057f9a5af4833aeac9992203a223c, user_id=03244087f1a34bddb11ea46aba11a148, role_id=68d3a95a9fc34fd1a28a9861d22ee828)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:01.627520 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649cc934-4c35-4785-9606-f189567ca368 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:01.633300 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:01.703787 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 546/1084] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 17 16:42:01 2022] PUT /identity/v3/projects/927057f9a5af4833aeac9992203a223c/users/03244087f1a34bddb11ea46aba11a148/roles/68d3a95a9fc34fd1a28a9861d22ee828 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 17 16:42:03.488741 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaae381-3bec-46aa-a62d-d015eaf76bb0 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:03.489322 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaae381-3bec-46aa-a62d-d015eaf76bb0 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:03.489691 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaae381-3bec-46aa-a62d-d015eaf76bb0 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:03.492229 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 539/1085] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:42:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:42:03.499966 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4732e-1376-404b-95f8-da91631a99ea None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:03.500502 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4732e-1376-404b-95f8-da91631a99ea None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:03.500976 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4732e-1376-404b-95f8-da91631a99ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:03.522827 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-adb4732e-1376-404b-95f8-da91631a99ea None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:42:03.524047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-adb4732e-1376-404b-95f8-da91631a99ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:42:03.599423 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 547/1086] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:42:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:42:03.624152 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaae381-3bec-46aa-a62d-d015eaf76bb0 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:03.718809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:03.742617 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaae381-3bec-46aa-a62d-d015eaf76bb0 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:03.795928 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12437ecf-2f5d-4f25-9bf2-f5d5887cdfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:03.797261 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12437ecf-2f5d-4f25-9bf2-f5d5887cdfc2 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:03.797713 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12437ecf-2f5d-4f25-9bf2-f5d5887cdfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:03.798226 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12437ecf-2f5d-4f25-9bf2-f5d5887cdfc2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:03.819893 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12437ecf-2f5d-4f25-9bf2-f5d5887cdfc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:42:03.835098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-12437ecf-2f5d-4f25-9bf2-f5d5887cdfc2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:42:03.835098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:42:03.835098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:42:03.835098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:42:03.835098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:42:03.835098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:42:03.835098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:42:03.835098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:42:03.835098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:42:03.835098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:42:03.835098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:42:03.835098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:42:03.835098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:42:03.835098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:42:03.835098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:42:03.835098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:42:03.837202 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:42:03.837202 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:42:03.837202 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:42:03.837202 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:42:03.837202 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:42:03.837202 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:42:03.837202 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:42:03.837202 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:42:03.837202 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:42:03.837202 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:42:03.837202 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:42:03.837202 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:42:03.837202 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:42:03.837202 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:42:03.837202 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:42:03.837202 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:42:03.838494 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:42:03.838494 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:42:03.838494 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:42:03.838494 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:42:03.838494 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:42:03.838494 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:42:03.838494 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:42:03.838494 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:42:03.838494 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:42:03.838494 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:42:03.838494 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:42:03.838494 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:42:03.840178 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcfdd7-8374-451b-ae7f-b7f88a2d8c3e None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:03.840178 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcfdd7-8374-451b-ae7f-b7f88a2d8c3e None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:03.840178 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcfdd7-8374-451b-ae7f-b7f88a2d8c3e None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:03.840524 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 540/1087] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:42:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:42:03.842331 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 548/1088] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:42:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:42:03.844501 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12437ecf-2f5d-4f25-9bf2-f5d5887cdfc2 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:03.847970 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37181ead-e1c4-4d03-b2e9-54995ff5ecd8 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:03.848253 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37181ead-e1c4-4d03-b2e9-54995ff5ecd8 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:03.848531 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37181ead-e1c4-4d03-b2e9-54995ff5ecd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:03.865764 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-37181ead-e1c4-4d03-b2e9-54995ff5ecd8 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:42:03.866978 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-37181ead-e1c4-4d03-b2e9-54995ff5ecd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:42:03.956309 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 549/1089] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:42:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 17 16:42:03.965316 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12437ecf-2f5d-4f25-9bf2-f5d5887cdfc2 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:03.980200 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37181ead-e1c4-4d03-b2e9-54995ff5ecd8 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:04.011163 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60dfad0-7b11-4888-b69b-5915779a40a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:04.011790 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60dfad0-7b11-4888-b69b-5915779a40a6 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:04.012077 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60dfad0-7b11-4888-b69b-5915779a40a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:04.012362 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60dfad0-7b11-4888-b69b-5915779a40a6 None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:04.013426 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60dfad0-7b11-4888-b69b-5915779a40a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:42:04.013741 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60dfad0-7b11-4888-b69b-5915779a40a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:04.015206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60dfad0-7b11-4888-b69b-5915779a40a6 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:04.025576 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 541/1090] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:42:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:42:04.033876 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60dfad0-7b11-4888-b69b-5915779a40a6 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:04.037381 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:04.061290 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:04.117301 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37181ead-e1c4-4d03-b2e9-54995ff5ecd8 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:04.164481 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e377e-adb4-4d89-97ae-1dc6f68d02d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:04.165564 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e377e-adb4-4d89-97ae-1dc6f68d02d4 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:04.165998 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e377e-adb4-4d89-97ae-1dc6f68d02d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:04.166404 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e377e-adb4-4d89-97ae-1dc6f68d02d4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:04.168539 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60dfad0-7b11-4888-b69b-5915779a40a6 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:04.179745 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e377e-adb4-4d89-97ae-1dc6f68d02d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:42:04.189419 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-987e377e-adb4-4d89-97ae-1dc6f68d02d4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 17 16:42:04.189419 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:42:04.189419 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:42:04.189419 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:42:04.189419 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:42:04.189419 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:42:04.189419 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:42:04.189419 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:42:04.189419 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:42:04.189419 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:42:04.189419 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:42:04.189419 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:42:04.189419 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:42:04.189419 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:42:04.189419 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:42:04.189419 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:42:04.191982 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:42:04.191982 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:42:04.191982 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:42:04.191982 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:42:04.191982 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:42:04.191982 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:42:04.191982 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:42:04.191982 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:42:04.191982 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:42:04.191982 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:42:04.191982 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:42:04.191982 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:42:04.191982 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:42:04.191982 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:42:04.191982 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:42:04.191982 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:42:04.194047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:42:04.194047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:42:04.194047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:42:04.194047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:42:04.194047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:42:04.194047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:42:04.194047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:42:04.194047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:42:04.194047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:42:04.194047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:42:04.194047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 17 16:42:04.194047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:42:04.194047 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 550/1091] 10.222.0.40 () {56 vars in 1120 bytes} [Sun Jul 17 16:42:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:42:04.200995 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e377e-adb4-4d89-97ae-1dc6f68d02d4 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:04.232071 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5eeec-24d3-4ddc-adb2-851c123feaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:04.233648 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5eeec-24d3-4ddc-adb2-851c123feaf4 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:04.234135 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5eeec-24d3-4ddc-adb2-851c123feaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:04.234512 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5eeec-24d3-4ddc-adb2-851c123feaf4 None admin] PATH_INFO: `/v3/users/738cc6e7b90343cdbdff27d64e3f98ea` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:04.236528 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f5eeec-24d3-4ddc-adb2-851c123feaf4 None admin] RBAC: Authorizing `identity:get_user(user_id=738cc6e7b90343cdbdff27d64e3f98ea)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:04.239109 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f5eeec-24d3-4ddc-adb2-851c123feaf4 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:04.247865 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 542/1092] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:42:04 2022] GET /identity/v3/users/738cc6e7b90343cdbdff27d64e3f98ea => generated 285 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:04.249561 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5eeec-24d3-4ddc-adb2-851c123feaf4 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:04.345906 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:04.353778 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e377e-adb4-4d89-97ae-1dc6f68d02d4 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:04.381194 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:04.391184 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5eeec-24d3-4ddc-adb2-851c123feaf4 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:04.408774 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d711d-8b2e-476f-83de-02696ebe7d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:04.409757 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d711d-8b2e-476f-83de-02696ebe7d78 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:04.410085 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d711d-8b2e-476f-83de-02696ebe7d78 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:04.410476 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d711d-8b2e-476f-83de-02696ebe7d78 None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:04.411374 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d711d-8b2e-476f-83de-02696ebe7d78 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:42:04.411783 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d711d-8b2e-476f-83de-02696ebe7d78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:04.413543 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d711d-8b2e-476f-83de-02696ebe7d78 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:04.425113 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 551/1093] 10.222.0.40 () {56 vars in 1115 bytes} [Sun Jul 17 16:42:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:42:04.430689 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d711d-8b2e-476f-83de-02696ebe7d78 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:04.438306 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b44fa8-878b-47b6-af0d-db7c372fe2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:04.438974 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b44fa8-878b-47b6-af0d-db7c372fe2e9 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:04.439257 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b44fa8-878b-47b6-af0d-db7c372fe2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:04.439534 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b44fa8-878b-47b6-af0d-db7c372fe2e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:04.533680 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b44fa8-878b-47b6-af0d-db7c372fe2e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:04.536456 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b44fa8-878b-47b6-af0d-db7c372fe2e9 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:04.550199 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d711d-8b2e-476f-83de-02696ebe7d78 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:04.591761 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8411ad0-f04a-4b68-bc91-f46a9f0dbeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:04.592756 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8411ad0-f04a-4b68-bc91-f46a9f0dbeeb None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:04.593246 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8411ad0-f04a-4b68-bc91-f46a9f0dbeeb None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:04.593650 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8411ad0-f04a-4b68-bc91-f46a9f0dbeeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:04.609928 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8411ad0-f04a-4b68-bc91-f46a9f0dbeeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:04.609928 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8411ad0-f04a-4b68-bc91-f46a9f0dbeeb None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:04.627704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-c8411ad0-f04a-4b68-bc91-f46a9f0dbeeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:42:04.627704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:42:04.627704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:42:04.627704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:42:04.627704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:42:04.627704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:42:04.627704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:42:04.627704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:42:04.627704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:42:04.627704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:42:04.627704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:42:04.627704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:42:04.627704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:42:04.627704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:42:04.627704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:42:04.629553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:42:04.629553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:42:04.629553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:42:04.629553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:42:04.629553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:42:04.629553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:42:04.629553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:42:04.629553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:42:04.629553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:42:04.629553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:42:04.629553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:42:04.629553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:42:04.629553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:42:04.629553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:42:04.629553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:42:04.629553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:42:04.629553 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:42:04.631331 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:42:04.631331 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:42:04.631331 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:42:04.631331 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:42:04.631331 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 552/1094] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:42:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:42:04.631865 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:04.637219 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8411ad0-f04a-4b68-bc91-f46a9f0dbeeb None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:04.673010 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:04.678935 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 543/1095] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:42:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 17 16:42:04.686300 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b44fa8-878b-47b6-af0d-db7c372fe2e9 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:04.766272 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8411ad0-f04a-4b68-bc91-f46a9f0dbeeb None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:04.807855 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525da78-7adc-48de-b4e7-3aa5965f362b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:04.809373 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525da78-7adc-48de-b4e7-3aa5965f362b None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:04.809373 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525da78-7adc-48de-b4e7-3aa5965f362b None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:04.809828 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525da78-7adc-48de-b4e7-3aa5965f362b None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:04.810629 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d525da78-7adc-48de-b4e7-3aa5965f362b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:42:04.811023 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d525da78-7adc-48de-b4e7-3aa5965f362b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:04.812848 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d525da78-7adc-48de-b4e7-3aa5965f362b None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:04.826656 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 553/1096] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:42:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:42:04.830500 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b44fa8-878b-47b6-af0d-db7c372fe2e9 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:04.832776 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525da78-7adc-48de-b4e7-3aa5965f362b None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:04.870636 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7975e992-ebef-4fee-8122-c5347accac72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:04.872031 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975e992-ebef-4fee-8122-c5347accac72 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:04.872312 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975e992-ebef-4fee-8122-c5347accac72 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:04.872597 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975e992-ebef-4fee-8122-c5347accac72 None admin] PATH_INFO: `/v3/projects/1efc8a7f7d1d47b9955fad77a83260b7` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:04.873903 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7975e992-ebef-4fee-8122-c5347accac72 None admin] RBAC: Authorizing `identity:get_project(project_id=1efc8a7f7d1d47b9955fad77a83260b7)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:04.875361 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7975e992-ebef-4fee-8122-c5347accac72 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:04.877171 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 544/1097] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:42:04 2022] GET /identity/v3/projects/1efc8a7f7d1d47b9955fad77a83260b7 => generated 311 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:42:04.883260 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7975e992-ebef-4fee-8122-c5347accac72 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:04.951495 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:04.954183 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:04.962181 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525da78-7adc-48de-b4e7-3aa5965f362b None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:05.000661 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab506d-c56c-4a30-b671-20e6cb14b264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:05.001629 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab506d-c56c-4a30-b671-20e6cb14b264 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:05.001916 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab506d-c56c-4a30-b671-20e6cb14b264 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:05.002191 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab506d-c56c-4a30-b671-20e6cb14b264 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:05.010146 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ab506d-c56c-4a30-b671-20e6cb14b264 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:05.011608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ab506d-c56c-4a30-b671-20e6cb14b264 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:05.027001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-c5ab506d-c56c-4a30-b671-20e6cb14b264 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 17 16:42:05.027001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:42:05.027001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:42:05.027001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:42:05.027001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:42:05.027001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:42:05.027001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:42:05.027001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:42:05.027001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:42:05.027001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:42:05.027001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:42:05.027001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:42:05.027001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:42:05.027001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:42:05.027001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:42:05.027001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:42:05.028958 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:42:05.028958 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:42:05.028958 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:42:05.028958 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:42:05.028958 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:42:05.028958 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:42:05.028958 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:42:05.028958 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:42:05.028958 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:42:05.028958 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:42:05.028958 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:42:05.028958 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:42:05.028958 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:42:05.028958 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:42:05.028958 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:42:05.028958 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:42:05.028958 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:42:05.028958 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:42:05.030831 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:42:05.030831 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:42:05.030831 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:42:05.030831 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:42:05.030831 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:42:05.030831 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:42:05.030831 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:42:05.030831 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:42:05.030831 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 17 16:42:05.030831 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:42:05.030831 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 554/1098] 10.222.0.40 () {56 vars in 1093 bytes} [Sun Jul 17 16:42:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:42:05.034330 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab506d-c56c-4a30-b671-20e6cb14b264 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:05.050023 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7975e992-ebef-4fee-8122-c5347accac72 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:05.086579 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aeb4bc-3f5c-49c5-9821-93bd4928df04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:05.087250 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aeb4bc-3f5c-49c5-9821-93bd4928df04 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:05.087548 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aeb4bc-3f5c-49c5-9821-93bd4928df04 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:05.087828 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aeb4bc-3f5c-49c5-9821-93bd4928df04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:05.088681 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aeb4bc-3f5c-49c5-9821-93bd4928df04 None admin] RBAC: Adding query filter params (role.id=1025051975f64f689effa58dc4259e36, scope.project.id=1efc8a7f7d1d47b9955fad77a83260b7, user.id=738cc6e7b90343cdbdff27d64e3f98ea) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:42:05.089045 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aeb4bc-3f5c-49c5-9821-93bd4928df04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:05.091282 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aeb4bc-3f5c-49c5-9821-93bd4928df04 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:05.097800 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 545/1099] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:42:04 2022] GET /identity/v3/role_assignments?user.id=738cc6e7b90343cdbdff27d64e3f98ea&scope.project.id=1efc8a7f7d1d47b9955fad77a83260b7&role.id=1025051975f64f689effa58dc4259e36 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:05.204757 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab506d-c56c-4a30-b671-20e6cb14b264 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:05.248249 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de2396a-4a9b-4140-8ab5-93ddc19a2727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:05.248983 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2396a-4a9b-4140-8ab5-93ddc19a2727 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:05.249273 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2396a-4a9b-4140-8ab5-93ddc19a2727 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:05.249556 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de2396a-4a9b-4140-8ab5-93ddc19a2727 None admin] PATH_INFO: `/v3/users` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:05.250441 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de2396a-4a9b-4140-8ab5-93ddc19a2727 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:42:05.250758 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de2396a-4a9b-4140-8ab5-93ddc19a2727 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:05.252258 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de2396a-4a9b-4140-8ab5-93ddc19a2727 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:05.273020 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 555/1100] 10.222.0.40 () {56 vars in 1133 bytes} [Sun Jul 17 16:42:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:42:05.279900 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aeb4bc-3f5c-49c5-9821-93bd4928df04 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:05.412309 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aeb4bc-3f5c-49c5-9821-93bd4928df04 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:05.414164 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:05.453007 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7aaad4-702a-4505-b0e4-361a106f5cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:05.453588 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7aaad4-702a-4505-b0e4-361a106f5cba None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:05.453893 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7aaad4-702a-4505-b0e4-361a106f5cba None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:05.454185 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7aaad4-702a-4505-b0e4-361a106f5cba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:05.539726 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7aaad4-702a-4505-b0e4-361a106f5cba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:05.541536 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7aaad4-702a-4505-b0e4-361a106f5cba None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:05.654907 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 546/1101] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:42:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 17 16:42:05.661234 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de2396a-4a9b-4140-8ab5-93ddc19a2727 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:05.674491 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:05.786437 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de2396a-4a9b-4140-8ab5-93ddc19a2727 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:05.827246 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae3ee9-86e3-4fc3-bbef-65554f1e77a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:05.828072 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae3ee9-86e3-4fc3-bbef-65554f1e77a0 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:05.828360 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae3ee9-86e3-4fc3-bbef-65554f1e77a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:05.828647 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae3ee9-86e3-4fc3-bbef-65554f1e77a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:05.840428 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eae3ee9-86e3-4fc3-bbef-65554f1e77a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:05.841936 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eae3ee9-86e3-4fc3-bbef-65554f1e77a0 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:05.861464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-9eae3ee9-86e3-4fc3-bbef-65554f1e77a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:42:05.861464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:42:05.861464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:42:05.861464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:42:05.861464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:42:05.861464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:42:05.861464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:42:05.861464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:42:05.861464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:42:05.861464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:42:05.861464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:42:05.861464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:42:05.861464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:42:05.861464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:42:05.861464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:42:05.862608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:42:05.862608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:42:05.862608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:42:05.862608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:42:05.862608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:42:05.862608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:42:05.862608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:42:05.862608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:42:05.862608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:42:05.862608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:42:05.862608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:42:05.862608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:42:05.862608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:42:05.862608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:42:05.862608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:42:05.862608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:42:05.862608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:42:05.863498 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:42:05.863498 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:42:05.863498 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:42:05.863498 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:42:05.863498 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 556/1102] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:42:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:42:05.868804 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7aaad4-702a-4505-b0e4-361a106f5cba None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:05.871659 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:05.988576 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7aaad4-702a-4505-b0e4-361a106f5cba None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:06.029503 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47827232-2b99-414c-b685-b24489d190da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:06.030202 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47827232-2b99-414c-b685-b24489d190da None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:06.030492 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47827232-2b99-414c-b685-b24489d190da None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:06.030775 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47827232-2b99-414c-b685-b24489d190da None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:06.031536 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47827232-2b99-414c-b685-b24489d190da None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:42:06.031852 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47827232-2b99-414c-b685-b24489d190da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:06.033355 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47827232-2b99-414c-b685-b24489d190da None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:06.048268 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 547/1103] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:42:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:06.054769 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae3ee9-86e3-4fc3-bbef-65554f1e77a0 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:06.124798 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:06.171539 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae3ee9-86e3-4fc3-bbef-65554f1e77a0 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:06.227162 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5c4cf-96d3-4e0d-8b57-dc66e0b0dbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:06.228543 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5c4cf-96d3-4e0d-8b57-dc66e0b0dbbc None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:06.229090 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5c4cf-96d3-4e0d-8b57-dc66e0b0dbbc None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:06.229608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5c4cf-96d3-4e0d-8b57-dc66e0b0dbbc None admin] PATH_INFO: `/v3/projects/service` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:06.242915 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e5c4cf-96d3-4e0d-8b57-dc66e0b0dbbc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:06.246470 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e5c4cf-96d3-4e0d-8b57-dc66e0b0dbbc None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:06.275773 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-00e5c4cf-96d3-4e0d-8b57-dc66e0b0dbbc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:42:06.275773 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:42:06.275773 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:42:06.275773 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:42:06.275773 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:42:06.275773 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:42:06.275773 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:42:06.275773 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:42:06.275773 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:42:06.275773 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:42:06.275773 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:42:06.275773 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:42:06.275773 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:42:06.275773 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:42:06.275773 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:42:06.278142 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:42:06.278142 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:42:06.278142 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:42:06.278142 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:42:06.278142 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:42:06.278142 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:42:06.278142 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:42:06.278142 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:42:06.278142 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:42:06.278142 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:42:06.278142 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:42:06.278142 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:42:06.278142 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:42:06.278142 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:42:06.278142 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:42:06.278142 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:42:06.278142 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:42:06.279869 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:42:06.279869 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:42:06.279869 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:42:06.279869 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:42:06.279869 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:42:06.279869 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:42:06.279869 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:42:06.279869 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:42:06.279869 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 557/1104] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:42:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:42:06.286174 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47827232-2b99-414c-b685-b24489d190da None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:06.359354 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:06.406186 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47827232-2b99-414c-b685-b24489d190da None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:06.452679 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6309bdc-a60f-4d25-80c5-ed0898ee1718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:06.453504 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6309bdc-a60f-4d25-80c5-ed0898ee1718 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:06.453891 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6309bdc-a60f-4d25-80c5-ed0898ee1718 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:06.454258 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6309bdc-a60f-4d25-80c5-ed0898ee1718 None admin] PATH_INFO: `/v3/projects` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:06.455195 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6309bdc-a60f-4d25-80c5-ed0898ee1718 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:42:06.455601 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6309bdc-a60f-4d25-80c5-ed0898ee1718 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:06.457279 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6309bdc-a60f-4d25-80c5-ed0898ee1718 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:06.471390 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 548/1105] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:42:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:42:06.477941 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5c4cf-96d3-4e0d-8b57-dc66e0b0dbbc None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:06.653619 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5c4cf-96d3-4e0d-8b57-dc66e0b0dbbc None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:06.691958 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:06.715957 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481f8e4-8b91-46b8-bbb2-7f78250f1e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:06.717154 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481f8e4-8b91-46b8-bbb2-7f78250f1e67 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:06.717662 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481f8e4-8b91-46b8-bbb2-7f78250f1e67 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:06.718153 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481f8e4-8b91-46b8-bbb2-7f78250f1e67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:06.719752 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0481f8e4-8b91-46b8-bbb2-7f78250f1e67 None admin] RBAC: Adding query filter params (role.id=68d3a95a9fc34fd1a28a9861d22ee828, scope.project.id=927057f9a5af4833aeac9992203a223c, user.id=03244087f1a34bddb11ea46aba11a148) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:42:06.720313 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0481f8e4-8b91-46b8-bbb2-7f78250f1e67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:06.723567 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0481f8e4-8b91-46b8-bbb2-7f78250f1e67 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:06.744824 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 558/1106] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:42:06 2022] GET /identity/v3/role_assignments?user.id=03244087f1a34bddb11ea46aba11a148&scope.project.id=927057f9a5af4833aeac9992203a223c&role.id=68d3a95a9fc34fd1a28a9861d22ee828 => generated 608 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:07.633148 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a41deb-3731-4bf2-9ec4-e69684e4d9f7 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:07.633933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a41deb-3731-4bf2-9ec4-e69684e4d9f7 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:07.634086 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a41deb-3731-4bf2-9ec4-e69684e4d9f7 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:07.635833 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 549/1107] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:42:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:42:07.642800 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b04d61-fc81-488a-8995-3aa05da99867 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:07.643235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b04d61-fc81-488a-8995-3aa05da99867 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:07.646521 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b04d61-fc81-488a-8995-3aa05da99867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:07.665761 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-99b04d61-fc81-488a-8995-3aa05da99867 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:42:07.667120 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-99b04d61-fc81-488a-8995-3aa05da99867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:42:07.782812 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 559/1108] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:42:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 17 16:42:07.809447 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a41deb-3731-4bf2-9ec4-e69684e4d9f7 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:07.930010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a41deb-3731-4bf2-9ec4-e69684e4d9f7 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:07.958256 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:07.970603 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adf576f-49da-44ef-9ab1-7b29812065cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:07.972888 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf576f-49da-44ef-9ab1-7b29812065cb None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:07.972888 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf576f-49da-44ef-9ab1-7b29812065cb None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:07.972888 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf576f-49da-44ef-9ab1-7b29812065cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:07.973894 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adf576f-49da-44ef-9ab1-7b29812065cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:07.975421 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adf576f-49da-44ef-9ab1-7b29812065cb None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:07.977549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 550/1109] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:42:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:07.984884 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b04d61-fc81-488a-8995-3aa05da99867 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:08.104289 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b04d61-fc81-488a-8995-3aa05da99867 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:08.120344 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:08.142975 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fb1c1f-f3e1-47ff-ab51-9f7ddc9605f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:08.143880 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb1c1f-f3e1-47ff-ab51-9f7ddc9605f3 None admin] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:08.144205 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb1c1f-f3e1-47ff-ab51-9f7ddc9605f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:08.144524 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb1c1f-f3e1-47ff-ab51-9f7ddc9605f3 None admin] PATH_INFO: `/v3/users` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:08.145499 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fb1c1f-f3e1-47ff-ab51-9f7ddc9605f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:08.147009 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fb1c1f-f3e1-47ff-ab51-9f7ddc9605f3 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:08.259861 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 560/1110] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 17 16:42:07 2022] POST /identity/v3/users => generated 286 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 17 16:42:10.527476 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e84790-a604-4ff1-a06b-5c8210b7dc30 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:10.528036 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e84790-a604-4ff1-a06b-5c8210b7dc30 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:10.528327 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e84790-a604-4ff1-a06b-5c8210b7dc30 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:10.529992 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 551/1111] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:42:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:42:10.535598 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde8d81-bcdd-4107-95d6-80892bfd0e01 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:10.536081 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde8d81-bcdd-4107-95d6-80892bfd0e01 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:10.536170 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde8d81-bcdd-4107-95d6-80892bfd0e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:10.554445 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-bfde8d81-bcdd-4107-95d6-80892bfd0e01 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:42:10.555670 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-bfde8d81-bcdd-4107-95d6-80892bfd0e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:42:10.625982 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 561/1112] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:42:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:42:10.650239 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e84790-a604-4ff1-a06b-5c8210b7dc30 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:10.757930 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e84790-a604-4ff1-a06b-5c8210b7dc30 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:10.795595 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc011d42-a1df-4b0c-8878-f66ccfe0f061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:10.796378 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc011d42-a1df-4b0c-8878-f66ccfe0f061 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:10.796664 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc011d42-a1df-4b0c-8878-f66ccfe0f061 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:10.796971 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc011d42-a1df-4b0c-8878-f66ccfe0f061 None admin] PATH_INFO: `/v3/roles/71cf67eb04704663aff719f6e3804f1a` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:10.798482 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc011d42-a1df-4b0c-8878-f66ccfe0f061 None admin] RBAC: Authorizing `identity:get_role(role_id=71cf67eb04704663aff719f6e3804f1a)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:10.799861 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc011d42-a1df-4b0c-8878-f66ccfe0f061 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:10.801287 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 552/1113] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:42:10 2022] GET /identity/v3/roles/71cf67eb04704663aff719f6e3804f1a => generated 223 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:10.808457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde8d81-bcdd-4107-95d6-80892bfd0e01 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:10.910616 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:10.913783 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde8d81-bcdd-4107-95d6-80892bfd0e01 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:10.955628 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ce5e1-df5c-4f9c-97c8-937401770bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:10.956590 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ce5e1-df5c-4f9c-97c8-937401770bea None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:10.956979 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ce5e1-df5c-4f9c-97c8-937401770bea None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:10.957450 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ce5e1-df5c-4f9c-97c8-937401770bea None admin] PATH_INFO: `/v3/users/660149f35ebd44ec8f4659314fe1ee66` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:10.974741 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47ce5e1-df5c-4f9c-97c8-937401770bea None admin] RBAC: Authorizing `identity:get_user(user_id=660149f35ebd44ec8f4659314fe1ee66)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:10.974741 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47ce5e1-df5c-4f9c-97c8-937401770bea None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:10.977987 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 562/1114] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:42:10 2022] GET /identity/v3/users/660149f35ebd44ec8f4659314fe1ee66 => generated 286 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:42:10.987295 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc011d42-a1df-4b0c-8878-f66ccfe0f061 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:11.053664 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:11.098270 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc011d42-a1df-4b0c-8878-f66ccfe0f061 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:11.134342 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd4f53-2c05-4611-a922-574cb6596d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:11.134990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd4f53-2c05-4611-a922-574cb6596d4a None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:11.135281 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd4f53-2c05-4611-a922-574cb6596d4a None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:11.135556 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd4f53-2c05-4611-a922-574cb6596d4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:11.208713 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd4f53-2c05-4611-a922-574cb6596d4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:11.210202 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd4f53-2c05-4611-a922-574cb6596d4a None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:11.269452 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:11.313080 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 553/1115] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:42:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 17 16:42:11.320500 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ce5e1-df5c-4f9c-97c8-937401770bea None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:11.425994 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ce5e1-df5c-4f9c-97c8-937401770bea None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:11.462083 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08042d0-41a3-44de-b217-fd20afe9092e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:11.462944 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08042d0-41a3-44de-b217-fd20afe9092e None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:11.463223 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08042d0-41a3-44de-b217-fd20afe9092e None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:11.463492 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08042d0-41a3-44de-b217-fd20afe9092e None admin] PATH_INFO: `/v3/projects/1efc8a7f7d1d47b9955fad77a83260b7` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:11.465037 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08042d0-41a3-44de-b217-fd20afe9092e None admin] RBAC: Authorizing `identity:get_project(project_id=1efc8a7f7d1d47b9955fad77a83260b7)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:11.466494 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08042d0-41a3-44de-b217-fd20afe9092e None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:11.468217 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 563/1116] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:42:11 2022] GET /identity/v3/projects/1efc8a7f7d1d47b9955fad77a83260b7 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:11.474431 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd4f53-2c05-4611-a922-574cb6596d4a None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:11.580815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd4f53-2c05-4611-a922-574cb6596d4a None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:11.622561 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47067e-7c30-4af4-986e-e04228450a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:11.623263 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47067e-7c30-4af4-986e-e04228450a3a None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:11.623549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47067e-7c30-4af4-986e-e04228450a3a None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:11.623831 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47067e-7c30-4af4-986e-e04228450a3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:11.624716 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb47067e-7c30-4af4-986e-e04228450a3a None admin] RBAC: Adding query filter params (role.id=71cf67eb04704663aff719f6e3804f1a, scope.project.id=1efc8a7f7d1d47b9955fad77a83260b7, user.id=660149f35ebd44ec8f4659314fe1ee66) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:42:11.625071 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb47067e-7c30-4af4-986e-e04228450a3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:11.626596 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb47067e-7c30-4af4-986e-e04228450a3a None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:11.634082 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 554/1117] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:42:11 2022] GET /identity/v3/role_assignments?user.id=660149f35ebd44ec8f4659314fe1ee66&scope.project.id=1efc8a7f7d1d47b9955fad77a83260b7&role.id=71cf67eb04704663aff719f6e3804f1a => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:13.717226 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50211b1-1227-4424-99c6-d5125a1fddeb None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:13.717839 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50211b1-1227-4424-99c6-d5125a1fddeb None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:13.717839 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50211b1-1227-4424-99c6-d5125a1fddeb None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:13.719151 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 564/1118] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:42:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:42:13.725151 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544d350-9864-457f-860f-cf443c29c6e7 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:13.725596 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544d350-9864-457f-860f-cf443c29c6e7 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:13.725795 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544d350-9864-457f-860f-cf443c29c6e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:13.743625 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-c544d350-9864-457f-860f-cf443c29c6e7 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:42:13.744986 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-c544d350-9864-457f-860f-cf443c29c6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:42:13.765753 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:13.828382 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 555/1119] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:42:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:42:13.851893 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50211b1-1227-4424-99c6-d5125a1fddeb None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:13.883610 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:13.990841 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50211b1-1227-4424-99c6-d5125a1fddeb None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:14.030488 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b48fb-9db5-4b12-8e83-8ef70e387ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:14.031364 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b48fb-9db5-4b12-8e83-8ef70e387ff3 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:14.031756 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b48fb-9db5-4b12-8e83-8ef70e387ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:14.032092 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b48fb-9db5-4b12-8e83-8ef70e387ff3 None admin] PATH_INFO: `/v3/roles/71cf67eb04704663aff719f6e3804f1a` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:14.033860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b48fb-9db5-4b12-8e83-8ef70e387ff3 None admin] RBAC: Authorizing `identity:get_role(role_id=71cf67eb04704663aff719f6e3804f1a)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:14.035302 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b48fb-9db5-4b12-8e83-8ef70e387ff3 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:14.036805 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 565/1120] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:42:13 2022] GET /identity/v3/roles/71cf67eb04704663aff719f6e3804f1a => generated 223 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:42:14.043714 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c544d350-9864-457f-860f-cf443c29c6e7 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:14.152986 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c544d350-9864-457f-860f-cf443c29c6e7 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:14.192454 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68407ffb-54fd-4489-b5a0-71f3da5e1f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:14.193451 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68407ffb-54fd-4489-b5a0-71f3da5e1f4e None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:14.193779 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68407ffb-54fd-4489-b5a0-71f3da5e1f4e None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:14.194124 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68407ffb-54fd-4489-b5a0-71f3da5e1f4e None admin] PATH_INFO: `/v3/users/660149f35ebd44ec8f4659314fe1ee66` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:14.195400 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68407ffb-54fd-4489-b5a0-71f3da5e1f4e None admin] RBAC: Authorizing `identity:get_user(user_id=660149f35ebd44ec8f4659314fe1ee66)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:14.196951 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68407ffb-54fd-4489-b5a0-71f3da5e1f4e None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:14.198723 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 556/1121] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:42:14 2022] GET /identity/v3/users/660149f35ebd44ec8f4659314fe1ee66 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:14.204798 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b48fb-9db5-4b12-8e83-8ef70e387ff3 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:14.305664 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:14.313395 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b48fb-9db5-4b12-8e83-8ef70e387ff3 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:14.350470 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a344cfe-a1f8-479c-9d94-a5d71c283165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:14.351347 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a344cfe-a1f8-479c-9d94-a5d71c283165 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:14.351636 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a344cfe-a1f8-479c-9d94-a5d71c283165 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:14.351915 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a344cfe-a1f8-479c-9d94-a5d71c283165 None admin] PATH_INFO: `/v3/projects/1efc8a7f7d1d47b9955fad77a83260b7` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:14.353167 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a344cfe-a1f8-479c-9d94-a5d71c283165 None admin] RBAC: Authorizing `identity:get_project(project_id=1efc8a7f7d1d47b9955fad77a83260b7)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:14.354644 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a344cfe-a1f8-479c-9d94-a5d71c283165 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:14.356576 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 566/1122] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:42:14 2022] GET /identity/v3/projects/1efc8a7f7d1d47b9955fad77a83260b7 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:42:14.362823 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68407ffb-54fd-4489-b5a0-71f3da5e1f4e None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:14.386827 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:14.473930 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68407ffb-54fd-4489-b5a0-71f3da5e1f4e None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:14.512718 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01650af1-2703-4426-9886-a5d26f817ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:14.513721 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01650af1-2703-4426-9886-a5d26f817ddb None admin] REQUEST_METHOD: `PUT` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:14.514058 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01650af1-2703-4426-9886-a5d26f817ddb None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:14.514377 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01650af1-2703-4426-9886-a5d26f817ddb None admin] PATH_INFO: `/v3/projects/1efc8a7f7d1d47b9955fad77a83260b7/users/660149f35ebd44ec8f4659314fe1ee66/roles/71cf67eb04704663aff719f6e3804f1a` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:14.516672 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01650af1-2703-4426-9886-a5d26f817ddb None admin] RBAC: Authorizing `identity:create_grant(project_id=1efc8a7f7d1d47b9955fad77a83260b7, user_id=660149f35ebd44ec8f4659314fe1ee66, role_id=71cf67eb04704663aff719f6e3804f1a)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:14.518149 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01650af1-2703-4426-9886-a5d26f817ddb None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:14.591392 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 557/1123] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 17 16:42:14 2022] PUT /identity/v3/projects/1efc8a7f7d1d47b9955fad77a83260b7/users/660149f35ebd44ec8f4659314fe1ee66/roles/71cf67eb04704663aff719f6e3804f1a => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 17 16:42:16.671271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc71e5-b042-4b7d-9221-67f5020f75e6 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:16.671853 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc71e5-b042-4b7d-9221-67f5020f75e6 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:16.671853 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dc71e5-b042-4b7d-9221-67f5020f75e6 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:16.673533 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 567/1124] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:42:16.679394 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906f996-8102-46f7-b7a1-f82e1d1e88e0 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:16.679966 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906f996-8102-46f7-b7a1-f82e1d1e88e0 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:16.679966 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906f996-8102-46f7-b7a1-f82e1d1e88e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:16.697716 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-f906f996-8102-46f7-b7a1-f82e1d1e88e0 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:42:16.699040 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-f906f996-8102-46f7-b7a1-f82e1d1e88e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:42:16.778500 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 558/1125] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:42:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:42:16.803381 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc71e5-b042-4b7d-9221-67f5020f75e6 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:16.926698 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dc71e5-b042-4b7d-9221-67f5020f75e6 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:16.968269 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd105987-4244-41bd-a72e-a4e4de2bd1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:16.969267 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd105987-4244-41bd-a72e-a4e4de2bd1bd None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:16.969556 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd105987-4244-41bd-a72e-a4e4de2bd1bd None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:16.969836 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd105987-4244-41bd-a72e-a4e4de2bd1bd None admin] PATH_INFO: `/v3/roles/71cf67eb04704663aff719f6e3804f1a` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:16.971491 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd105987-4244-41bd-a72e-a4e4de2bd1bd None admin] RBAC: Authorizing `identity:get_role(role_id=71cf67eb04704663aff719f6e3804f1a)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:16.972984 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd105987-4244-41bd-a72e-a4e4de2bd1bd None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:16.974482 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 568/1126] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:42:16 2022] GET /identity/v3/roles/71cf67eb04704663aff719f6e3804f1a => generated 223 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:42:16.981941 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906f996-8102-46f7-b7a1-f82e1d1e88e0 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:16.994820 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:17.096089 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906f996-8102-46f7-b7a1-f82e1d1e88e0 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:17.136414 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e26c1-f2fb-451a-b477-5568639b220a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:17.137351 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e26c1-f2fb-451a-b477-5568639b220a None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:17.137631 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e26c1-f2fb-451a-b477-5568639b220a None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:17.137908 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220e26c1-f2fb-451a-b477-5568639b220a None admin] PATH_INFO: `/v3/users/660149f35ebd44ec8f4659314fe1ee66` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:17.139214 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220e26c1-f2fb-451a-b477-5568639b220a None admin] RBAC: Authorizing `identity:get_user(user_id=660149f35ebd44ec8f4659314fe1ee66)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:17.140683 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220e26c1-f2fb-451a-b477-5568639b220a None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:17.142444 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 559/1127] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:42:16 2022] GET /identity/v3/users/660149f35ebd44ec8f4659314fe1ee66 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:17.148805 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd105987-4244-41bd-a72e-a4e4de2bd1bd None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:17.189341 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:17.309396 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd105987-4244-41bd-a72e-a4e4de2bd1bd None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:17.367423 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f175b50-a496-4361-8874-36a20ad08739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:17.368525 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f175b50-a496-4361-8874-36a20ad08739 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:17.369063 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f175b50-a496-4361-8874-36a20ad08739 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:17.369554 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f175b50-a496-4361-8874-36a20ad08739 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:17.488428 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f175b50-a496-4361-8874-36a20ad08739 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:17.491156 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f175b50-a496-4361-8874-36a20ad08739 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:17.544838 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:17.673701 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 569/1128] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:42:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 17 16:42:17.675145 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e26c1-f2fb-451a-b477-5568639b220a None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:17.760323 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:17.783301 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220e26c1-f2fb-451a-b477-5568639b220a None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:17.821631 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412b076-2bea-4539-bd88-e63b79dc9303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:17.822631 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412b076-2bea-4539-bd88-e63b79dc9303 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:17.822961 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412b076-2bea-4539-bd88-e63b79dc9303 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:17.823771 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412b076-2bea-4539-bd88-e63b79dc9303 None admin] PATH_INFO: `/v3/projects/1efc8a7f7d1d47b9955fad77a83260b7` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:17.825189 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d412b076-2bea-4539-bd88-e63b79dc9303 None admin] RBAC: Authorizing `identity:get_project(project_id=1efc8a7f7d1d47b9955fad77a83260b7)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:17.826724 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d412b076-2bea-4539-bd88-e63b79dc9303 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:17.828532 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 560/1129] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:42:17 2022] GET /identity/v3/projects/1efc8a7f7d1d47b9955fad77a83260b7 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:42:17.835606 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f175b50-a496-4361-8874-36a20ad08739 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:17.996661 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f175b50-a496-4361-8874-36a20ad08739 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:18.027215 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:18.033660 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb6414-1af4-4000-b3ae-5f98df87438f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:18.034396 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb6414-1af4-4000-b3ae-5f98df87438f None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:18.034681 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb6414-1af4-4000-b3ae-5f98df87438f None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:18.034961 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb6414-1af4-4000-b3ae-5f98df87438f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:18.035811 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb6414-1af4-4000-b3ae-5f98df87438f None admin] RBAC: Adding query filter params (role.id=71cf67eb04704663aff719f6e3804f1a, scope.project.id=1efc8a7f7d1d47b9955fad77a83260b7, user.id=660149f35ebd44ec8f4659314fe1ee66) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:42:18.036126 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb6414-1af4-4000-b3ae-5f98df87438f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:18.037604 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb6414-1af4-4000-b3ae-5f98df87438f None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:18.044291 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 570/1130] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:42:17 2022] GET /identity/v3/role_assignments?user.id=660149f35ebd44ec8f4659314fe1ee66&scope.project.id=1efc8a7f7d1d47b9955fad77a83260b7&role.id=71cf67eb04704663aff719f6e3804f1a => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:20.208372 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6933b-cc93-4b20-a92c-0a0508def057 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:20.208947 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6933b-cc93-4b20-a92c-0a0508def057 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:20.208947 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6933b-cc93-4b20-a92c-0a0508def057 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:20.210348 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 561/1131] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:42:20.217096 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fbe76f-eb4c-46a0-9b47-c9e66fbb3f65 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:20.217866 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fbe76f-eb4c-46a0-9b47-c9e66fbb3f65 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:20.218015 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fbe76f-eb4c-46a0-9b47-c9e66fbb3f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:20.235291 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-15fbe76f-eb4c-46a0-9b47-c9e66fbb3f65 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:42:20.236546 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-15fbe76f-eb4c-46a0-9b47-c9e66fbb3f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:42:20.308241 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 571/1132] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:42:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:42:20.332031 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6933b-cc93-4b20-a92c-0a0508def057 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:20.443612 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6933b-cc93-4b20-a92c-0a0508def057 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:20.479639 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2319d2f3-a290-4ae9-9acc-bd46e25dee46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:20.480660 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319d2f3-a290-4ae9-9acc-bd46e25dee46 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:20.481007 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319d2f3-a290-4ae9-9acc-bd46e25dee46 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:20.481328 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319d2f3-a290-4ae9-9acc-bd46e25dee46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:20.482629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2319d2f3-a290-4ae9-9acc-bd46e25dee46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:20.484167 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2319d2f3-a290-4ae9-9acc-bd46e25dee46 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:20.485893 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 562/1133] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:42:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:20.495524 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fbe76f-eb4c-46a0-9b47-c9e66fbb3f65 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:20.607908 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fbe76f-eb4c-46a0-9b47-c9e66fbb3f65 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:20.636412 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:20.647601 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a0af0b-eefe-4afa-9169-ec1b35b9082b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:20.648394 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0af0b-eefe-4afa-9169-ec1b35b9082b None admin] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:20.648765 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0af0b-eefe-4afa-9169-ec1b35b9082b None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:20.649171 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0af0b-eefe-4afa-9169-ec1b35b9082b None admin] PATH_INFO: `/v3/projects` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:20.649992 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a0af0b-eefe-4afa-9169-ec1b35b9082b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:20.652650 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a0af0b-eefe-4afa-9169-ec1b35b9082b None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:20.780967 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 572/1134] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Jul 17 16:42:20 2022] POST /identity/v3/projects => generated 311 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:42:22.896907 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41826840-6526-450e-9dd0-af247c24b027 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:22.897485 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41826840-6526-450e-9dd0-af247c24b027 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:22.897485 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41826840-6526-450e-9dd0-af247c24b027 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:22.898990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 563/1135] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:42:22.905338 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069e83b-56dc-4ee3-82da-f56e541e3a69 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:22.906051 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069e83b-56dc-4ee3-82da-f56e541e3a69 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:22.906261 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069e83b-56dc-4ee3-82da-f56e541e3a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:22.924690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-0069e83b-56dc-4ee3-82da-f56e541e3a69 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:42:22.925942 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-0069e83b-56dc-4ee3-82da-f56e541e3a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:42:22.995923 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 573/1136] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:42:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:42:23.020989 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41826840-6526-450e-9dd0-af247c24b027 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:23.038507 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:23.131916 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41826840-6526-450e-9dd0-af247c24b027 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:23.169731 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee5846-8138-49f1-96cf-cc3dcbe550a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:23.170595 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee5846-8138-49f1-96cf-cc3dcbe550a0 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:23.170981 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee5846-8138-49f1-96cf-cc3dcbe550a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:23.171343 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee5846-8138-49f1-96cf-cc3dcbe550a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:23.172671 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abee5846-8138-49f1-96cf-cc3dcbe550a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:23.174259 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abee5846-8138-49f1-96cf-cc3dcbe550a0 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:23.180728 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 564/1137] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:42:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:23.182300 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069e83b-56dc-4ee3-82da-f56e541e3a69 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:23.288044 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069e83b-56dc-4ee3-82da-f56e541e3a69 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:23.296005 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:23.325716 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1531e958-4b82-4f20-ad21-e7da591f51ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:23.326441 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531e958-4b82-4f20-ad21-e7da591f51ca None admin] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:23.326762 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531e958-4b82-4f20-ad21-e7da591f51ca None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:23.327079 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531e958-4b82-4f20-ad21-e7da591f51ca None admin] PATH_INFO: `/v3/users` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:23.327833 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1531e958-4b82-4f20-ad21-e7da591f51ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:23.329434 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1531e958-4b82-4f20-ad21-e7da591f51ca None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:23.410439 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 574/1138] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 17 16:42:23 2022] POST /identity/v3/users => generated 286 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:42:25.603448 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488165b-0ac5-414e-aa2d-747754fbfc5b None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:25.604586 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488165b-0ac5-414e-aa2d-747754fbfc5b None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:25.604729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488165b-0ac5-414e-aa2d-747754fbfc5b None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:25.606183 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 565/1139] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:42:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:42:25.612264 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fabbd0-bd2b-4aea-b7ba-5d5e6938f4ae None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:25.612829 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fabbd0-bd2b-4aea-b7ba-5d5e6938f4ae None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:25.612945 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fabbd0-bd2b-4aea-b7ba-5d5e6938f4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:25.631370 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-33fabbd0-bd2b-4aea-b7ba-5d5e6938f4ae None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:42:25.632610 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-33fabbd0-bd2b-4aea-b7ba-5d5e6938f4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:42:25.706756 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 575/1140] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:42:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:42:25.732369 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488165b-0ac5-414e-aa2d-747754fbfc5b None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:25.820930 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:25.841495 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488165b-0ac5-414e-aa2d-747754fbfc5b None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:25.879343 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751069b-27d3-497a-9f3a-a14098434a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:25.880042 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751069b-27d3-497a-9f3a-a14098434a3f None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:25.880334 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751069b-27d3-497a-9f3a-a14098434a3f None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:25.880610 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751069b-27d3-497a-9f3a-a14098434a3f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:25.891827 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f751069b-27d3-497a-9f3a-a14098434a3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:42:25.899647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-f751069b-27d3-497a-9f3a-a14098434a3f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:42:25.899647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:42:25.899647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:42:25.899647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:42:25.899647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:42:25.899647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:42:25.899647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:42:25.899647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:42:25.899647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:42:25.899647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:42:25.899647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:42:25.899647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:42:25.899647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:42:25.899647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:42:25.899647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:42:25.899647 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:42:25.901481 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:42:25.901481 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:42:25.901481 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:42:25.901481 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:42:25.901481 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:42:25.901481 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:42:25.901481 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:42:25.901481 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:42:25.901481 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:42:25.901481 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:42:25.901481 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:42:25.901481 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:42:25.901481 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:42:25.901481 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:42:25.901481 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:42:25.901481 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:42:25.902617 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:42:25.902617 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:42:25.902617 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:42:25.902617 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:42:25.902617 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:42:25.902617 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:42:25.902617 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:42:25.902617 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:42:25.902617 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:42:25.902617 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:42:25.902617 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:42:25.902617 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:42:25.902617 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 566/1141] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:42:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:42:25.910103 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fabbd0-bd2b-4aea-b7ba-5d5e6938f4ae None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:26.023000 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:26.033160 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fabbd0-bd2b-4aea-b7ba-5d5e6938f4ae None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:26.073471 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1518ba-7219-4539-8808-25dc15c08a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:26.074165 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1518ba-7219-4539-8808-25dc15c08a94 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:26.074455 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1518ba-7219-4539-8808-25dc15c08a94 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:26.074738 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1518ba-7219-4539-8808-25dc15c08a94 None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:26.075518 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1518ba-7219-4539-8808-25dc15c08a94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:42:26.075832 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1518ba-7219-4539-8808-25dc15c08a94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:26.077294 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1518ba-7219-4539-8808-25dc15c08a94 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:26.086810 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 576/1142] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:42:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:26.093946 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751069b-27d3-497a-9f3a-a14098434a3f None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:26.205599 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751069b-27d3-497a-9f3a-a14098434a3f None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:26.241162 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2481533-e899-4109-86c7-db5fea4a73b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:26.242045 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2481533-e899-4109-86c7-db5fea4a73b8 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:26.242339 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2481533-e899-4109-86c7-db5fea4a73b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:26.242620 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2481533-e899-4109-86c7-db5fea4a73b8 None admin] PATH_INFO: `/v3/users/60ceae510f6a4d38a90c504bfd6f295f` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:26.255461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:26.257965 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2481533-e899-4109-86c7-db5fea4a73b8 None admin] RBAC: Authorizing `identity:get_user(user_id=60ceae510f6a4d38a90c504bfd6f295f)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:26.259426 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2481533-e899-4109-86c7-db5fea4a73b8 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:26.261915 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 567/1143] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:42:26 2022] GET /identity/v3/users/60ceae510f6a4d38a90c504bfd6f295f => generated 286 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:42:26.271880 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1518ba-7219-4539-8808-25dc15c08a94 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:26.382113 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1518ba-7219-4539-8808-25dc15c08a94 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:26.419556 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf23be7d-51cd-4e5d-8533-15087730648c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:26.420220 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf23be7d-51cd-4e5d-8533-15087730648c None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:26.420506 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf23be7d-51cd-4e5d-8533-15087730648c None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:26.420820 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf23be7d-51cd-4e5d-8533-15087730648c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:26.447506 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:26.494168 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf23be7d-51cd-4e5d-8533-15087730648c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:26.496514 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf23be7d-51cd-4e5d-8533-15087730648c None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:26.601705 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 577/1144] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:42:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 17 16:42:26.608417 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2481533-e899-4109-86c7-db5fea4a73b8 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:26.720878 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2481533-e899-4109-86c7-db5fea4a73b8 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:26.758713 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0eff34-7341-4eef-9327-8ddcb6ff0c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:26.759598 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0eff34-7341-4eef-9327-8ddcb6ff0c6e None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:26.759894 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0eff34-7341-4eef-9327-8ddcb6ff0c6e None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:26.760921 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0eff34-7341-4eef-9327-8ddcb6ff0c6e None admin] PATH_INFO: `/v3/projects/7beaa77b77914cf3b0ec89b9fbb23c74` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:26.762222 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0eff34-7341-4eef-9327-8ddcb6ff0c6e None admin] RBAC: Authorizing `identity:get_project(project_id=7beaa77b77914cf3b0ec89b9fbb23c74)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:26.763688 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0eff34-7341-4eef-9327-8ddcb6ff0c6e None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:26.765520 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 568/1145] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:42:26 2022] GET /identity/v3/projects/7beaa77b77914cf3b0ec89b9fbb23c74 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:42:26.775422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf23be7d-51cd-4e5d-8533-15087730648c None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:26.834213 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:26.881206 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf23be7d-51cd-4e5d-8533-15087730648c None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:26.917653 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f7c61e-b6aa-4040-a961-e72b8a5ee83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:26.918317 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7c61e-b6aa-4040-a961-e72b8a5ee83c None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:26.918607 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7c61e-b6aa-4040-a961-e72b8a5ee83c None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:26.918888 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f7c61e-b6aa-4040-a961-e72b8a5ee83c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:26.919724 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7c61e-b6aa-4040-a961-e72b8a5ee83c None admin] RBAC: Adding query filter params (role.id=1025051975f64f689effa58dc4259e36, scope.project.id=7beaa77b77914cf3b0ec89b9fbb23c74, user.id=60ceae510f6a4d38a90c504bfd6f295f) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:42:26.920039 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7c61e-b6aa-4040-a961-e72b8a5ee83c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:26.921588 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f7c61e-b6aa-4040-a961-e72b8a5ee83c None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:26.927349 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 578/1146] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:42:26 2022] GET /identity/v3/role_assignments?user.id=60ceae510f6a4d38a90c504bfd6f295f&scope.project.id=7beaa77b77914cf3b0ec89b9fbb23c74&role.id=1025051975f64f689effa58dc4259e36 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:29.002407 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9d7c3-1e05-46a8-a211-90f08e1f5706 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:29.003001 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9d7c3-1e05-46a8-a211-90f08e1f5706 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:29.003001 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9d7c3-1e05-46a8-a211-90f08e1f5706 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:29.004582 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 569/1147] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:42:29.011146 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314ceef-dc9f-4d0f-8e2c-4766dee12fea None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:29.011700 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314ceef-dc9f-4d0f-8e2c-4766dee12fea None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:29.011700 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314ceef-dc9f-4d0f-8e2c-4766dee12fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:29.029354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-1314ceef-dc9f-4d0f-8e2c-4766dee12fea None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:42:29.030593 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-1314ceef-dc9f-4d0f-8e2c-4766dee12fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:42:29.104658 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 579/1148] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:42:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 17 16:42:29.126985 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d9d7c3-1e05-46a8-a211-90f08e1f5706 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:29.199896 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:29.236272 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d9d7c3-1e05-46a8-a211-90f08e1f5706 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:29.273623 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e0a39c-9fb3-473c-ae79-656834e2abd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:29.274426 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0a39c-9fb3-473c-ae79-656834e2abd7 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:29.274714 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0a39c-9fb3-473c-ae79-656834e2abd7 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:29.274996 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0a39c-9fb3-473c-ae79-656834e2abd7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:29.285540 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e0a39c-9fb3-473c-ae79-656834e2abd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:42:29.293613 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-63e0a39c-9fb3-473c-ae79-656834e2abd7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:42:29.293613 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:42:29.293613 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:42:29.293613 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:42:29.293613 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:42:29.293613 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:42:29.293613 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:42:29.293613 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:42:29.293613 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:42:29.293613 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:42:29.293613 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:42:29.293613 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:42:29.293613 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:42:29.293613 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:42:29.293613 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:42:29.293613 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:42:29.295314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:42:29.295314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:42:29.295314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:42:29.295314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:42:29.295314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:42:29.295314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:42:29.295314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:42:29.295314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:42:29.295314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:42:29.295314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:42:29.295314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:42:29.295314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:42:29.295314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:42:29.295314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:42:29.295314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:42:29.295314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:42:29.296947 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:42:29.296947 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:42:29.296947 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:42:29.296947 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:42:29.296947 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:42:29.296947 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:42:29.296947 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:42:29.296947 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:42:29.296947 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:42:29.296947 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:42:29.296947 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:42:29.296947 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:42:29.296947 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 570/1149] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:42:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:42:29.302505 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314ceef-dc9f-4d0f-8e2c-4766dee12fea None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:29.356958 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:29.411867 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314ceef-dc9f-4d0f-8e2c-4766dee12fea None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:29.453971 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1d60f-6234-402d-8f62-d5dd8b7b33a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:29.454916 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1d60f-6234-402d-8f62-d5dd8b7b33a5 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:29.455207 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1d60f-6234-402d-8f62-d5dd8b7b33a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:29.455490 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1d60f-6234-402d-8f62-d5dd8b7b33a5 None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:29.456280 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d1d60f-6234-402d-8f62-d5dd8b7b33a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:42:29.456592 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d1d60f-6234-402d-8f62-d5dd8b7b33a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:29.458044 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d1d60f-6234-402d-8f62-d5dd8b7b33a5 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:29.467821 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 580/1150] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:42:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:29.477065 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e0a39c-9fb3-473c-ae79-656834e2abd7 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:29.652892 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e0a39c-9fb3-473c-ae79-656834e2abd7 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:29.702430 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:29.715980 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4fb0aa-4bf8-4ea9-b40c-143a8a57e6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:29.717541 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4fb0aa-4bf8-4ea9-b40c-143a8a57e6e6 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:29.718044 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4fb0aa-4bf8-4ea9-b40c-143a8a57e6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:29.718552 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4fb0aa-4bf8-4ea9-b40c-143a8a57e6e6 None admin] PATH_INFO: `/v3/users/60ceae510f6a4d38a90c504bfd6f295f` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:29.720639 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4fb0aa-4bf8-4ea9-b40c-143a8a57e6e6 None admin] RBAC: Authorizing `identity:get_user(user_id=60ceae510f6a4d38a90c504bfd6f295f)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:29.723414 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4fb0aa-4bf8-4ea9-b40c-143a8a57e6e6 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:29.726492 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 571/1151] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:42:29 2022] GET /identity/v3/users/60ceae510f6a4d38a90c504bfd6f295f => generated 286 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:29.732741 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1d60f-6234-402d-8f62-d5dd8b7b33a5 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:29.839265 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1d60f-6234-402d-8f62-d5dd8b7b33a5 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:29.850454 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:29.875158 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53458fa1-2964-46df-9c1e-2206a2f69b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:29.876093 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53458fa1-2964-46df-9c1e-2206a2f69b9c None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:29.876423 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53458fa1-2964-46df-9c1e-2206a2f69b9c None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:29.876746 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53458fa1-2964-46df-9c1e-2206a2f69b9c None admin] PATH_INFO: `/v3/projects/7beaa77b77914cf3b0ec89b9fbb23c74` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:29.878059 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53458fa1-2964-46df-9c1e-2206a2f69b9c None admin] RBAC: Authorizing `identity:get_project(project_id=7beaa77b77914cf3b0ec89b9fbb23c74)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:29.879644 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53458fa1-2964-46df-9c1e-2206a2f69b9c None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:29.881468 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 581/1152] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:42:29 2022] GET /identity/v3/projects/7beaa77b77914cf3b0ec89b9fbb23c74 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:42:29.888562 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4fb0aa-4bf8-4ea9-b40c-143a8a57e6e6 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:30.002618 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4fb0aa-4bf8-4ea9-b40c-143a8a57e6e6 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:30.039668 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7b1d1-19ab-4e63-a08e-a0710550c4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:30.040616 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7b1d1-19ab-4e63-a08e-a0710550c4ec None admin] REQUEST_METHOD: `PUT` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:30.040980 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7b1d1-19ab-4e63-a08e-a0710550c4ec None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:30.041868 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7b1d1-19ab-4e63-a08e-a0710550c4ec None admin] PATH_INFO: `/v3/projects/7beaa77b77914cf3b0ec89b9fbb23c74/users/60ceae510f6a4d38a90c504bfd6f295f/roles/1025051975f64f689effa58dc4259e36` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:30.044047 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c7b1d1-19ab-4e63-a08e-a0710550c4ec None admin] RBAC: Authorizing `identity:create_grant(project_id=7beaa77b77914cf3b0ec89b9fbb23c74, user_id=60ceae510f6a4d38a90c504bfd6f295f, role_id=1025051975f64f689effa58dc4259e36)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:30.045567 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c7b1d1-19ab-4e63-a08e-a0710550c4ec None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:30.139677 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 572/1153] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 17 16:42:29 2022] PUT /identity/v3/projects/7beaa77b77914cf3b0ec89b9fbb23c74/users/60ceae510f6a4d38a90c504bfd6f295f/roles/1025051975f64f689effa58dc4259e36 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 17 16:42:32.200509 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d366c7e-40d9-48e6-8979-838d23bb0168 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:32.201164 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d366c7e-40d9-48e6-8979-838d23bb0168 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:32.201164 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d366c7e-40d9-48e6-8979-838d23bb0168 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:32.202559 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 582/1154] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:42:32.208846 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5c9c2-a283-43af-9374-4ae3ee26a98e None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:32.209440 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5c9c2-a283-43af-9374-4ae3ee26a98e None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:32.209440 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5c9c2-a283-43af-9374-4ae3ee26a98e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:32.227638 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-5aa5c9c2-a283-43af-9374-4ae3ee26a98e None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:42:32.228960 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-5aa5c9c2-a283-43af-9374-4ae3ee26a98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:42:32.285682 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:32.304175 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 573/1155] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:42:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:42:32.328702 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d366c7e-40d9-48e6-8979-838d23bb0168 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:32.435783 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d366c7e-40d9-48e6-8979-838d23bb0168 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:32.470947 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441eab12-b253-4bb4-8c65-98865f745c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:32.471744 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441eab12-b253-4bb4-8c65-98865f745c08 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:32.472029 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441eab12-b253-4bb4-8c65-98865f745c08 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:32.472306 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441eab12-b253-4bb4-8c65-98865f745c08 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:32.482490 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441eab12-b253-4bb4-8c65-98865f745c08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:42:32.490109 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-441eab12-b253-4bb4-8c65-98865f745c08 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:42:32.490109 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:42:32.490109 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:42:32.490109 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:42:32.490109 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:42:32.490109 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:42:32.490109 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:42:32.490109 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:42:32.490109 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:42:32.490109 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:42:32.490109 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:42:32.490109 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:42:32.490109 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:42:32.490109 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:42:32.490109 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:42:32.490109 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:42:32.491217 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:42:32.491217 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:42:32.491217 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:42:32.491217 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:42:32.491217 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:42:32.491217 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:42:32.491217 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:42:32.491217 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:42:32.491217 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:42:32.491217 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:42:32.491217 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:42:32.491217 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:42:32.491217 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:42:32.491217 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:42:32.491217 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:42:32.491217 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:42:32.492188 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:42:32.492188 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:42:32.492188 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:42:32.492188 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:42:32.492188 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:42:32.492188 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:42:32.492188 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:42:32.492188 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:42:32.492188 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:42:32.492188 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:42:32.492188 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:42:32.492188 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:42:32.492188 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 583/1156] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:42:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:42:32.498231 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5c9c2-a283-43af-9374-4ae3ee26a98e None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:32.604968 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5c9c2-a283-43af-9374-4ae3ee26a98e None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:32.640876 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1903da44-8fd3-4bc4-b3da-8fc1d21d8f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:32.641559 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903da44-8fd3-4bc4-b3da-8fc1d21d8f4a None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:32.641849 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903da44-8fd3-4bc4-b3da-8fc1d21d8f4a None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:32.642133 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1903da44-8fd3-4bc4-b3da-8fc1d21d8f4a None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:32.642917 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1903da44-8fd3-4bc4-b3da-8fc1d21d8f4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:42:32.643213 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1903da44-8fd3-4bc4-b3da-8fc1d21d8f4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:32.644604 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1903da44-8fd3-4bc4-b3da-8fc1d21d8f4a None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:32.654081 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 574/1157] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:42:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:32.660501 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441eab12-b253-4bb4-8c65-98865f745c08 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:32.702408 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:32.763674 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441eab12-b253-4bb4-8c65-98865f745c08 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:32.801083 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb3af2-503e-4ed1-aacc-ede2dc839646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:32.801965 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb3af2-503e-4ed1-aacc-ede2dc839646 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:32.802254 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb3af2-503e-4ed1-aacc-ede2dc839646 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:32.802534 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb3af2-503e-4ed1-aacc-ede2dc839646 None admin] PATH_INFO: `/v3/users/60ceae510f6a4d38a90c504bfd6f295f` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:32.803729 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb3af2-503e-4ed1-aacc-ede2dc839646 None admin] RBAC: Authorizing `identity:get_user(user_id=60ceae510f6a4d38a90c504bfd6f295f)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:32.805198 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb3af2-503e-4ed1-aacc-ede2dc839646 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:32.807106 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 584/1158] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:42:32 2022] GET /identity/v3/users/60ceae510f6a4d38a90c504bfd6f295f => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:32.816276 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1903da44-8fd3-4bc4-b3da-8fc1d21d8f4a None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:32.867002 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:32.919370 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1903da44-8fd3-4bc4-b3da-8fc1d21d8f4a None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:32.953114 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb98ca1a-3b8d-40c8-b8d5-0231d3582f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:32.953677 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb98ca1a-3b8d-40c8-b8d5-0231d3582f63 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:32.953962 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb98ca1a-3b8d-40c8-b8d5-0231d3582f63 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:32.954243 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb98ca1a-3b8d-40c8-b8d5-0231d3582f63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:33.024805 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb98ca1a-3b8d-40c8-b8d5-0231d3582f63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:33.026500 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb98ca1a-3b8d-40c8-b8d5-0231d3582f63 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:33.075402 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:33.127678 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 575/1159] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:42:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 17 16:42:33.133641 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb3af2-503e-4ed1-aacc-ede2dc839646 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:33.236145 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb3af2-503e-4ed1-aacc-ede2dc839646 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:33.243715 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:33.270439 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca662c27-9aa8-4e90-acc8-bf56c0ec3bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:33.271302 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca662c27-9aa8-4e90-acc8-bf56c0ec3bfa None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:33.271592 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca662c27-9aa8-4e90-acc8-bf56c0ec3bfa None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:33.271873 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca662c27-9aa8-4e90-acc8-bf56c0ec3bfa None admin] PATH_INFO: `/v3/projects/7beaa77b77914cf3b0ec89b9fbb23c74` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:33.273082 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca662c27-9aa8-4e90-acc8-bf56c0ec3bfa None admin] RBAC: Authorizing `identity:get_project(project_id=7beaa77b77914cf3b0ec89b9fbb23c74)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:33.274523 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca662c27-9aa8-4e90-acc8-bf56c0ec3bfa None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:33.276937 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 585/1160] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:42:33 2022] GET /identity/v3/projects/7beaa77b77914cf3b0ec89b9fbb23c74 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:33.283184 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb98ca1a-3b8d-40c8-b8d5-0231d3582f63 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:33.388368 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb98ca1a-3b8d-40c8-b8d5-0231d3582f63 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:33.423461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34155dcc-3e30-4387-9467-b37431b9ea3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:33.424224 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34155dcc-3e30-4387-9467-b37431b9ea3e None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:33.424515 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34155dcc-3e30-4387-9467-b37431b9ea3e None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:33.424815 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34155dcc-3e30-4387-9467-b37431b9ea3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:33.425663 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34155dcc-3e30-4387-9467-b37431b9ea3e None admin] RBAC: Adding query filter params (role.id=1025051975f64f689effa58dc4259e36, scope.project.id=7beaa77b77914cf3b0ec89b9fbb23c74, user.id=60ceae510f6a4d38a90c504bfd6f295f) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:42:33.425982 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34155dcc-3e30-4387-9467-b37431b9ea3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:33.427421 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34155dcc-3e30-4387-9467-b37431b9ea3e None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:33.433502 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 576/1161] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:42:33 2022] GET /identity/v3/role_assignments?user.id=60ceae510f6a4d38a90c504bfd6f295f&scope.project.id=7beaa77b77914cf3b0ec89b9fbb23c74&role.id=1025051975f64f689effa58dc4259e36 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:35.530564 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ce683-f894-45d2-a328-894639a87cf2 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:35.531106 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ce683-f894-45d2-a328-894639a87cf2 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:35.531106 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ce683-f894-45d2-a328-894639a87cf2 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:35.532489 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 586/1162] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:42:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:42:35.537863 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444e477-6857-4374-b479-396f929a72fb None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:35.538325 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444e477-6857-4374-b479-396f929a72fb None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:35.538414 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444e477-6857-4374-b479-396f929a72fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:35.558073 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-4444e477-6857-4374-b479-396f929a72fb None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:42:35.559299 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-4444e477-6857-4374-b479-396f929a72fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:42:35.569052 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:35.631528 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 577/1163] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:42:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:42:35.655559 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ce683-f894-45d2-a328-894639a87cf2 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:35.759580 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ce683-f894-45d2-a328-894639a87cf2 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:35.794446 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e07fe-6b64-49d7-9f5b-d5d0f8810bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:35.795021 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e07fe-6b64-49d7-9f5b-d5d0f8810bf8 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:35.795311 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e07fe-6b64-49d7-9f5b-d5d0f8810bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:35.795595 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e07fe-6b64-49d7-9f5b-d5d0f8810bf8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:35.839941 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:35.866926 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e07fe-6b64-49d7-9f5b-d5d0f8810bf8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:35.868456 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e07fe-6b64-49d7-9f5b-d5d0f8810bf8 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:35.967771 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 587/1164] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:42:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 17 16:42:35.974243 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444e477-6857-4374-b479-396f929a72fb None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:36.076116 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444e477-6857-4374-b479-396f929a72fb None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:36.110431 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d44bd67-a37c-4b93-9737-b4f26d516592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:36.111229 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44bd67-a37c-4b93-9737-b4f26d516592 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:36.111541 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44bd67-a37c-4b93-9737-b4f26d516592 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:36.111816 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44bd67-a37c-4b93-9737-b4f26d516592 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:36.118460 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d44bd67-a37c-4b93-9737-b4f26d516592 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:36.121091 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d44bd67-a37c-4b93-9737-b4f26d516592 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:36.136140 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-4d44bd67-a37c-4b93-9737-b4f26d516592 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 17 16:42:36.136140 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:42:36.136140 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 17 16:42:36.136140 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 17 16:42:36.136140 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:42:36.136140 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:42:36.136140 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:42:36.136140 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:42:36.136140 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 17 16:42:36.136140 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:42:36.136140 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 17 16:42:36.136140 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:42:36.136140 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:42:36.136140 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:42:36.136140 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:42:36.136140 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:42:36.136140 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:42:36.137440 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:42:36.137440 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:42:36.137440 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:42:36.137440 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:42:36.137440 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:42:36.137440 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:42:36.137440 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:42:36.137440 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:42:36.137440 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:42:36.137440 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:42:36.137440 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:42:36.137440 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:42:36.137440 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:42:36.137440 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:42:36.137440 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:42:36.137440 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:42:36.138346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:42:36.138346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:42:36.138346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:42:36.138346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:42:36.138346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:42:36.138346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:42:36.138346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:42:36.138346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:42:36.138346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:42:36.138346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:42:36.138346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 17 16:42:36.138346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:42:36.138346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 17 16:42:36.138346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:42:36.138346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 578/1165] 10.222.0.40 () {56 vars in 1117 bytes} [Sun Jul 17 16:42:35 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:42:36.143751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e07fe-6b64-49d7-9f5b-d5d0f8810bf8 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:36.225764 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:36.250965 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e07fe-6b64-49d7-9f5b-d5d0f8810bf8 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:36.287028 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2f731d-ebfb-4397-b6f3-a8f245c0b0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:36.287701 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f731d-ebfb-4397-b6f3-a8f245c0b0ce None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:36.287990 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f731d-ebfb-4397-b6f3-a8f245c0b0ce None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:36.288270 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2f731d-ebfb-4397-b6f3-a8f245c0b0ce None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:36.289026 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2f731d-ebfb-4397-b6f3-a8f245c0b0ce None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:42:36.289341 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2f731d-ebfb-4397-b6f3-a8f245c0b0ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:36.291293 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2f731d-ebfb-4397-b6f3-a8f245c0b0ce None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:36.298688 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 588/1166] 10.222.0.40 () {56 vars in 1115 bytes} [Sun Jul 17 16:42:36 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:38.352858 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e971989-555e-4a2b-8572-64bc7182b54e None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:38.353634 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e971989-555e-4a2b-8572-64bc7182b54e None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:38.353634 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e971989-555e-4a2b-8572-64bc7182b54e None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:38.354812 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 579/1167] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:42:38.360300 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d63294-16b1-44bf-b903-40985a0967c7 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:38.360771 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d63294-16b1-44bf-b903-40985a0967c7 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:38.360896 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d63294-16b1-44bf-b903-40985a0967c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:38.378124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-64d63294-16b1-44bf-b903-40985a0967c7 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:42:38.379334 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-64d63294-16b1-44bf-b903-40985a0967c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:42:38.451949 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 589/1168] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:42:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:42:38.474863 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e971989-555e-4a2b-8572-64bc7182b54e None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:38.478569 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:38.579704 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e971989-555e-4a2b-8572-64bc7182b54e None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:38.614321 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6497e83-948d-41fb-8cf8-bb11b057f2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:38.614991 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6497e83-948d-41fb-8cf8-bb11b057f2fa None admin] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:38.615276 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6497e83-948d-41fb-8cf8-bb11b057f2fa None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:38.615557 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6497e83-948d-41fb-8cf8-bb11b057f2fa None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:38.616304 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6497e83-948d-41fb-8cf8-bb11b057f2fa None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:38.617714 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6497e83-948d-41fb-8cf8-bb11b057f2fa None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:38.725967 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 580/1169] 10.222.0.40 () {60 vars in 1138 bytes} [Sun Jul 17 16:42:38 2022] POST /identity/v3/domains => generated 267 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:42:40.798109 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0889d-a1ad-43f5-86f5-535f8e7fdcc0 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:40.798692 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0889d-a1ad-43f5-86f5-535f8e7fdcc0 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:40.798692 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f0889d-a1ad-43f5-86f5-535f8e7fdcc0 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:40.800162 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 590/1170] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:42:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:42:40.806198 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09962481-db6a-4296-be62-3f0580737177 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:40.806779 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09962481-db6a-4296-be62-3f0580737177 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:40.806883 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09962481-db6a-4296-be62-3f0580737177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:40.826252 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-09962481-db6a-4296-be62-3f0580737177 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:42:40.827508 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-09962481-db6a-4296-be62-3f0580737177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:42:40.889250 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:40.900467 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 581/1171] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:42:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:42:40.923894 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f0889d-a1ad-43f5-86f5-535f8e7fdcc0 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:41.003017 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:41.046879 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f0889d-a1ad-43f5-86f5-535f8e7fdcc0 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:41.085014 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f153fce-5719-481c-9a6c-207640474170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:41.085814 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f153fce-5719-481c-9a6c-207640474170 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:41.086099 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f153fce-5719-481c-9a6c-207640474170 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:41.086378 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f153fce-5719-481c-9a6c-207640474170 None admin] PATH_INFO: `/v3/domains/ffc3eef992174ec0905a296c56fcad46` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:41.098978 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f153fce-5719-481c-9a6c-207640474170 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ffc3eef992174ec0905a296c56fcad46)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:41.100443 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f153fce-5719-481c-9a6c-207640474170 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:41.102244 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 591/1172] 10.222.0.40 () {56 vars in 1183 bytes} [Sun Jul 17 16:42:40 2022] GET /identity/v3/domains/ffc3eef992174ec0905a296c56fcad46 => generated 267 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:41.109260 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09962481-db6a-4296-be62-3f0580737177 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:41.216626 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09962481-db6a-4296-be62-3f0580737177 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:41.252684 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b04816-5e0d-412d-b328-c7b9a006c1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:41.253378 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b04816-5e0d-412d-b328-c7b9a006c1c7 None admin] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:41.253663 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b04816-5e0d-412d-b328-c7b9a006c1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:41.253944 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b04816-5e0d-412d-b328-c7b9a006c1c7 None admin] PATH_INFO: `/v3/projects` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:41.254659 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b04816-5e0d-412d-b328-c7b9a006c1c7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:41.256095 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b04816-5e0d-412d-b328-c7b9a006c1c7 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:41.331640 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 582/1173] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Jul 17 16:42:41 2022] POST /identity/v3/projects => generated 361 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:42:43.426396 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a8304-460e-41a2-aadd-bdf5151c98fb None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:43.427030 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a8304-460e-41a2-aadd-bdf5151c98fb None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:43.427030 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a8304-460e-41a2-aadd-bdf5151c98fb None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:43.428314 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 592/1174] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:42:43.433888 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be02e8-2caf-4cbd-a053-96020d3888dd None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:43.434396 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be02e8-2caf-4cbd-a053-96020d3888dd None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:43.434495 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be02e8-2caf-4cbd-a053-96020d3888dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:43.452425 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-c2be02e8-2caf-4cbd-a053-96020d3888dd None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:42:43.453672 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-c2be02e8-2caf-4cbd-a053-96020d3888dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:42:43.497244 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:43.524373 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 583/1175] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:42:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:42:43.549914 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a8304-460e-41a2-aadd-bdf5151c98fb None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:43.657756 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a8304-460e-41a2-aadd-bdf5151c98fb None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:43.695738 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:43.704669 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b7cc89-b3d6-4bc7-8f41-1102a199127e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:43.705617 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7cc89-b3d6-4bc7-8f41-1102a199127e None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:43.708335 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7cc89-b3d6-4bc7-8f41-1102a199127e None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:43.708335 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7cc89-b3d6-4bc7-8f41-1102a199127e None admin] PATH_INFO: `/v3/domains/ffc3eef992174ec0905a296c56fcad46` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:43.708335 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b7cc89-b3d6-4bc7-8f41-1102a199127e None admin] RBAC: Authorizing `identity:get_domain(domain_id=ffc3eef992174ec0905a296c56fcad46)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:43.709221 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b7cc89-b3d6-4bc7-8f41-1102a199127e None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:43.711120 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 593/1176] 10.222.0.40 () {56 vars in 1183 bytes} [Sun Jul 17 16:42:43 2022] GET /identity/v3/domains/ffc3eef992174ec0905a296c56fcad46 => generated 267 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:43.718818 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be02e8-2caf-4cbd-a053-96020d3888dd None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:43.838997 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be02e8-2caf-4cbd-a053-96020d3888dd None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:43.877157 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d9e08-55a1-46e0-8a29-7c1c50fc815b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:43.877841 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d9e08-55a1-46e0-8a29-7c1c50fc815b None admin] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:43.878126 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d9e08-55a1-46e0-8a29-7c1c50fc815b None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:43.878407 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d9e08-55a1-46e0-8a29-7c1c50fc815b None admin] PATH_INFO: `/v3/users` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:43.879130 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421d9e08-55a1-46e0-8a29-7c1c50fc815b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:43.880570 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421d9e08-55a1-46e0-8a29-7c1c50fc815b None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:43.975729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 584/1177] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 17 16:42:43 2022] POST /identity/v3/users => generated 311 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:42:46.096337 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e740a39-171c-45be-b467-0b34e8efb12f None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:46.097089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e740a39-171c-45be-b467-0b34e8efb12f None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:46.097089 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e740a39-171c-45be-b467-0b34e8efb12f None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:46.098332 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 594/1178] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:42:46.104414 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020da72c-173d-47da-9e6e-2ba0f07ed9dd None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:46.104933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020da72c-173d-47da-9e6e-2ba0f07ed9dd None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:46.105032 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020da72c-173d-47da-9e6e-2ba0f07ed9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:46.123812 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-020da72c-173d-47da-9e6e-2ba0f07ed9dd None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:42:46.125939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-020da72c-173d-47da-9e6e-2ba0f07ed9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:42:46.153691 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:46.199918 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 585/1179] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:42:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:42:46.223204 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e740a39-171c-45be-b467-0b34e8efb12f None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:46.328409 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e740a39-171c-45be-b467-0b34e8efb12f None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:46.363347 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9709ac80-6caa-47fe-91a7-bc1b6ef2f3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:46.364065 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709ac80-6caa-47fe-91a7-bc1b6ef2f3b2 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:46.364351 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709ac80-6caa-47fe-91a7-bc1b6ef2f3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:46.364631 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709ac80-6caa-47fe-91a7-bc1b6ef2f3b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:46.375718 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9709ac80-6caa-47fe-91a7-bc1b6ef2f3b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:42:46.383337 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-9709ac80-6caa-47fe-91a7-bc1b6ef2f3b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:42:46.383337 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:42:46.383337 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:42:46.383337 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:42:46.383337 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:42:46.383337 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:42:46.383337 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:42:46.383337 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:42:46.383337 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:42:46.383337 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:42:46.383337 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:42:46.383337 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:42:46.383337 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:42:46.383337 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:42:46.383337 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:42:46.383337 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:42:46.384551 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:42:46.384551 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:42:46.384551 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:42:46.384551 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:42:46.384551 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:42:46.384551 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:42:46.384551 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:42:46.384551 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:42:46.384551 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:42:46.384551 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:42:46.384551 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:42:46.384551 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:42:46.384551 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:42:46.384551 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:42:46.384551 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:42:46.384551 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:42:46.385578 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:42:46.385578 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:42:46.385578 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:42:46.385578 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:42:46.385578 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:42:46.385578 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:42:46.385578 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:42:46.385578 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:42:46.385578 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:42:46.385578 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:42:46.385578 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:42:46.385578 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:42:46.385578 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 595/1180] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:42:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:42:46.390929 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020da72c-173d-47da-9e6e-2ba0f07ed9dd None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:46.499435 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020da72c-173d-47da-9e6e-2ba0f07ed9dd None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:46.535865 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e98f1-ba7c-4e36-ad20-4fedc4f24780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:46.536537 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e98f1-ba7c-4e36-ad20-4fedc4f24780 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:46.536842 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e98f1-ba7c-4e36-ad20-4fedc4f24780 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:46.537120 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e98f1-ba7c-4e36-ad20-4fedc4f24780 None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:46.537868 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e98f1-ba7c-4e36-ad20-4fedc4f24780 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:42:46.538174 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e98f1-ba7c-4e36-ad20-4fedc4f24780 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:46.539541 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e98f1-ba7c-4e36-ad20-4fedc4f24780 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:46.548911 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 586/1181] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:42:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:46.556736 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9709ac80-6caa-47fe-91a7-bc1b6ef2f3b2 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:46.622535 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:46.668668 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9709ac80-6caa-47fe-91a7-bc1b6ef2f3b2 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:46.703153 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed051dce-70b5-4c39-ba30-247b6f736c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:46.704033 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed051dce-70b5-4c39-ba30-247b6f736c7d None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:46.704316 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed051dce-70b5-4c39-ba30-247b6f736c7d None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:46.704592 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed051dce-70b5-4c39-ba30-247b6f736c7d None admin] PATH_INFO: `/v3/users/c136c233a28c44ad987fdb0cda72bdac` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:46.719695 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed051dce-70b5-4c39-ba30-247b6f736c7d None admin] RBAC: Authorizing `identity:get_user(user_id=c136c233a28c44ad987fdb0cda72bdac)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:46.721207 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed051dce-70b5-4c39-ba30-247b6f736c7d None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:46.723049 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 596/1182] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:42:46 2022] GET /identity/v3/users/c136c233a28c44ad987fdb0cda72bdac => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:46.729954 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e98f1-ba7c-4e36-ad20-4fedc4f24780 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:46.759053 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:46.857208 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e98f1-ba7c-4e36-ad20-4fedc4f24780 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:46.897828 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e243a52-2648-4360-a59e-9fd78dcc3670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:46.898405 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e243a52-2648-4360-a59e-9fd78dcc3670 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:46.898746 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e243a52-2648-4360-a59e-9fd78dcc3670 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:46.899032 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e243a52-2648-4360-a59e-9fd78dcc3670 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:46.998729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e243a52-2648-4360-a59e-9fd78dcc3670 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:47.000212 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e243a52-2648-4360-a59e-9fd78dcc3670 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:47.027191 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:47.117790 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 587/1183] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:42:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 17 16:42:47.124584 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed051dce-70b5-4c39-ba30-247b6f736c7d None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:47.236686 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed051dce-70b5-4c39-ba30-247b6f736c7d None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:47.252432 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:47.276470 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc6e63-74b6-4d1b-b8a7-bf14d50bae41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:47.277383 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc6e63-74b6-4d1b-b8a7-bf14d50bae41 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:47.277674 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc6e63-74b6-4d1b-b8a7-bf14d50bae41 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:47.277956 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc6e63-74b6-4d1b-b8a7-bf14d50bae41 None admin] PATH_INFO: `/v3/projects/2e8284c9d9284b72a1b442e53a240f9a` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:47.279168 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc6e63-74b6-4d1b-b8a7-bf14d50bae41 None admin] RBAC: Authorizing `identity:get_project(project_id=2e8284c9d9284b72a1b442e53a240f9a)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:47.280621 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc6e63-74b6-4d1b-b8a7-bf14d50bae41 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:47.282408 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 597/1184] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:42:47 2022] GET /identity/v3/projects/2e8284c9d9284b72a1b442e53a240f9a => generated 361 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:47.288633 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e243a52-2648-4360-a59e-9fd78dcc3670 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:47.391104 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e243a52-2648-4360-a59e-9fd78dcc3670 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:47.416293 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:47.427837 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9bc3ef-81d4-4ca9-9210-9eb94833c72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:47.428546 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9bc3ef-81d4-4ca9-9210-9eb94833c72e None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:47.428799 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9bc3ef-81d4-4ca9-9210-9eb94833c72e None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:47.429534 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9bc3ef-81d4-4ca9-9210-9eb94833c72e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:47.429933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9bc3ef-81d4-4ca9-9210-9eb94833c72e None admin] RBAC: Adding query filter params (role.id=1025051975f64f689effa58dc4259e36, scope.project.id=2e8284c9d9284b72a1b442e53a240f9a, user.id=c136c233a28c44ad987fdb0cda72bdac) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:42:47.430249 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9bc3ef-81d4-4ca9-9210-9eb94833c72e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:47.432055 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9bc3ef-81d4-4ca9-9210-9eb94833c72e None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:47.437914 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 588/1185] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:42:47 2022] GET /identity/v3/role_assignments?user.id=c136c233a28c44ad987fdb0cda72bdac&scope.project.id=2e8284c9d9284b72a1b442e53a240f9a&role.id=1025051975f64f689effa58dc4259e36 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:49.487244 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579797e7-17bd-4214-b66e-06f0d1897cab None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:49.487772 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579797e7-17bd-4214-b66e-06f0d1897cab None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:49.487772 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579797e7-17bd-4214-b66e-06f0d1897cab None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:49.489131 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 598/1186] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:42:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:42:49.495693 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6511ca1-6f80-459a-bb00-02bc7decd05b None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:49.496203 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6511ca1-6f80-459a-bb00-02bc7decd05b None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:49.496358 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6511ca1-6f80-459a-bb00-02bc7decd05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:49.513746 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-d6511ca1-6f80-459a-bb00-02bc7decd05b None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:42:49.514969 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-d6511ca1-6f80-459a-bb00-02bc7decd05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:42:49.588849 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 589/1187] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:42:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:42:49.611245 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579797e7-17bd-4214-b66e-06f0d1897cab None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:49.721336 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579797e7-17bd-4214-b66e-06f0d1897cab None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:49.756763 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3a2df-5513-4733-964c-0321d9e4b8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:49.757507 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3a2df-5513-4733-964c-0321d9e4b8e3 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:49.757796 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3a2df-5513-4733-964c-0321d9e4b8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:49.758073 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3a2df-5513-4733-964c-0321d9e4b8e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:49.769240 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be3a2df-5513-4733-964c-0321d9e4b8e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:42:49.777509 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-6be3a2df-5513-4733-964c-0321d9e4b8e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:42:49.777509 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:42:49.777509 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:42:49.777509 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:42:49.777509 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:42:49.777509 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:42:49.777509 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:42:49.777509 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:42:49.777509 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:42:49.777509 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:42:49.777509 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:42:49.777509 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:42:49.777509 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:42:49.777509 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:42:49.777509 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:42:49.777509 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:42:49.778608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:42:49.778608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:42:49.778608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:42:49.778608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:42:49.778608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:42:49.778608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:42:49.778608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:42:49.778608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:42:49.778608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:42:49.778608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:42:49.778608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:42:49.778608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:42:49.778608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:42:49.778608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:42:49.778608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:42:49.778608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:42:49.779457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:42:49.779457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:42:49.779457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:42:49.779457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:42:49.779457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:42:49.779457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:42:49.779457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:42:49.779457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:42:49.779457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:42:49.779457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:42:49.779457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:42:49.779457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:42:49.779457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 599/1188] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:42:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:42:49.785101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6511ca1-6f80-459a-bb00-02bc7decd05b None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:49.891885 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6511ca1-6f80-459a-bb00-02bc7decd05b None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:49.909033 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:49.927316 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d75594-6717-4db3-a9b3-ac3e90bc03ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:49.927999 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d75594-6717-4db3-a9b3-ac3e90bc03ea None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:49.928283 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d75594-6717-4db3-a9b3-ac3e90bc03ea None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:49.928568 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d75594-6717-4db3-a9b3-ac3e90bc03ea None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:49.929361 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d75594-6717-4db3-a9b3-ac3e90bc03ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:42:49.929802 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d75594-6717-4db3-a9b3-ac3e90bc03ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:49.931809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d75594-6717-4db3-a9b3-ac3e90bc03ea None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:49.940616 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 590/1189] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:42:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:49.946569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3a2df-5513-4733-964c-0321d9e4b8e3 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:49.993716 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:50.051889 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3a2df-5513-4733-964c-0321d9e4b8e3 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:50.087514 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b9410-5621-4ca6-8603-33fb9dc9a6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:50.088400 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b9410-5621-4ca6-8603-33fb9dc9a6ef None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:50.088690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b9410-5621-4ca6-8603-33fb9dc9a6ef None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:50.089000 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b9410-5621-4ca6-8603-33fb9dc9a6ef None admin] PATH_INFO: `/v3/users/c136c233a28c44ad987fdb0cda72bdac` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:50.090233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516b9410-5621-4ca6-8603-33fb9dc9a6ef None admin] RBAC: Authorizing `identity:get_user(user_id=c136c233a28c44ad987fdb0cda72bdac)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:50.091678 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516b9410-5621-4ca6-8603-33fb9dc9a6ef None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:50.093382 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 600/1190] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:42:49 2022] GET /identity/v3/users/c136c233a28c44ad987fdb0cda72bdac => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:50.099365 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d75594-6717-4db3-a9b3-ac3e90bc03ea None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:50.206628 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d75594-6717-4db3-a9b3-ac3e90bc03ea None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:50.241391 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0589a65-77f1-4e72-aab5-917ae2606b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:50.242257 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0589a65-77f1-4e72-aab5-917ae2606b49 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:50.242545 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0589a65-77f1-4e72-aab5-917ae2606b49 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:50.242825 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0589a65-77f1-4e72-aab5-917ae2606b49 None admin] PATH_INFO: `/v3/projects/2e8284c9d9284b72a1b442e53a240f9a` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:50.244488 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0589a65-77f1-4e72-aab5-917ae2606b49 None admin] RBAC: Authorizing `identity:get_project(project_id=2e8284c9d9284b72a1b442e53a240f9a)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:50.245968 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0589a65-77f1-4e72-aab5-917ae2606b49 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:50.247697 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 591/1191] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:42:50 2022] GET /identity/v3/projects/2e8284c9d9284b72a1b442e53a240f9a => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:50.253933 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b9410-5621-4ca6-8603-33fb9dc9a6ef None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:50.357885 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b9410-5621-4ca6-8603-33fb9dc9a6ef None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:50.368748 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:50.392842 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603dadad-2806-4e60-91b5-867ed3472762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:50.393738 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603dadad-2806-4e60-91b5-867ed3472762 None admin] REQUEST_METHOD: `PUT` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:50.394011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603dadad-2806-4e60-91b5-867ed3472762 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:50.394284 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603dadad-2806-4e60-91b5-867ed3472762 None admin] PATH_INFO: `/v3/projects/2e8284c9d9284b72a1b442e53a240f9a/users/c136c233a28c44ad987fdb0cda72bdac/roles/1025051975f64f689effa58dc4259e36` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:50.396221 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603dadad-2806-4e60-91b5-867ed3472762 None admin] RBAC: Authorizing `identity:create_grant(project_id=2e8284c9d9284b72a1b442e53a240f9a, user_id=c136c233a28c44ad987fdb0cda72bdac, role_id=1025051975f64f689effa58dc4259e36)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:50.397699 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603dadad-2806-4e60-91b5-867ed3472762 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:50.471853 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 601/1192] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 17 16:42:50 2022] PUT /identity/v3/projects/2e8284c9d9284b72a1b442e53a240f9a/users/c136c233a28c44ad987fdb0cda72bdac/roles/1025051975f64f689effa58dc4259e36 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 17 16:42:52.526285 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a654ce-19b6-43b5-b80c-358e4a20d0f0 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:52.526822 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a654ce-19b6-43b5-b80c-358e4a20d0f0 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:52.526924 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a654ce-19b6-43b5-b80c-358e4a20d0f0 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:52.528206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 592/1193] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:42:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:42:52.533531 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac182e-80eb-49a5-a309-2b19c499f4a0 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:52.533930 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac182e-80eb-49a5-a309-2b19c499f4a0 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:52.534096 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac182e-80eb-49a5-a309-2b19c499f4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:52.551116 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-edac182e-80eb-49a5-a309-2b19c499f4a0 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:42:52.552316 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-edac182e-80eb-49a5-a309-2b19c499f4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:42:52.619805 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 602/1194] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:42:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:42:52.643320 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a654ce-19b6-43b5-b80c-358e4a20d0f0 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:52.692062 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:52.746898 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a654ce-19b6-43b5-b80c-358e4a20d0f0 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:52.782777 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6b3268-ac04-495b-9579-10a66faa98b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:52.784092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b3268-ac04-495b-9579-10a66faa98b7 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:52.784378 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b3268-ac04-495b-9579-10a66faa98b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:52.784656 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b3268-ac04-495b-9579-10a66faa98b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:52.796198 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6b3268-ac04-495b-9579-10a66faa98b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:42:52.803809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-fa6b3268-ac04-495b-9579-10a66faa98b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:42:52.803809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:42:52.803809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:42:52.803809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:42:52.803809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:42:52.803809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:42:52.803809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:42:52.803809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:42:52.803809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:42:52.803809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:42:52.803809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:42:52.803809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:42:52.803809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:42:52.803809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:42:52.803809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:42:52.803809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:42:52.804939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:42:52.804939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:42:52.804939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:42:52.804939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:42:52.804939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:42:52.804939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:42:52.804939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:42:52.804939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:42:52.804939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:42:52.804939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:42:52.804939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:42:52.804939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:42:52.804939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:42:52.804939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:42:52.804939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:42:52.804939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:42:52.805856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:42:52.805856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:42:52.805856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:42:52.805856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:42:52.805856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:42:52.805856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:42:52.805856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:42:52.805856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:42:52.805856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:42:52.805856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:42:52.805856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:42:52.805856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:42:52.805856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 593/1195] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:42:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:42:52.811554 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac182e-80eb-49a5-a309-2b19c499f4a0 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:52.909691 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:52.912755 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac182e-80eb-49a5-a309-2b19c499f4a0 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:52.949945 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df347c91-1b5a-41ae-8f68-d15c99540fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:52.950932 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df347c91-1b5a-41ae-8f68-d15c99540fb8 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:52.951226 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df347c91-1b5a-41ae-8f68-d15c99540fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:52.951678 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df347c91-1b5a-41ae-8f68-d15c99540fb8 None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:52.952767 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df347c91-1b5a-41ae-8f68-d15c99540fb8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:42:52.953404 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df347c91-1b5a-41ae-8f68-d15c99540fb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:52.954840 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df347c91-1b5a-41ae-8f68-d15c99540fb8 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:52.963936 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 603/1196] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:42:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:52.969811 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6b3268-ac04-495b-9579-10a66faa98b7 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:53.075993 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6b3268-ac04-495b-9579-10a66faa98b7 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:53.083022 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:53.110321 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2166a60c-5a8a-4cf8-a592-77368d895702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:53.111194 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2166a60c-5a8a-4cf8-a592-77368d895702 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:53.111482 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2166a60c-5a8a-4cf8-a592-77368d895702 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:53.111766 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2166a60c-5a8a-4cf8-a592-77368d895702 None admin] PATH_INFO: `/v3/users/c136c233a28c44ad987fdb0cda72bdac` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:53.113027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2166a60c-5a8a-4cf8-a592-77368d895702 None admin] RBAC: Authorizing `identity:get_user(user_id=c136c233a28c44ad987fdb0cda72bdac)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:53.114790 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2166a60c-5a8a-4cf8-a592-77368d895702 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:53.117043 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 594/1197] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 17 16:42:52 2022] GET /identity/v3/users/c136c233a28c44ad987fdb0cda72bdac => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:53.122626 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df347c91-1b5a-41ae-8f68-d15c99540fb8 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:53.225691 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df347c91-1b5a-41ae-8f68-d15c99540fb8 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:53.260101 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd08b28-55ed-4c7c-a108-7cb5b4482db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:53.260753 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd08b28-55ed-4c7c-a108-7cb5b4482db1 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:53.261072 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd08b28-55ed-4c7c-a108-7cb5b4482db1 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:53.261358 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd08b28-55ed-4c7c-a108-7cb5b4482db1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:53.290373 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:53.330518 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd08b28-55ed-4c7c-a108-7cb5b4482db1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:53.331899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd08b28-55ed-4c7c-a108-7cb5b4482db1 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:53.428493 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 604/1198] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:42:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 17 16:42:53.435297 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2166a60c-5a8a-4cf8-a592-77368d895702 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:53.536913 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2166a60c-5a8a-4cf8-a592-77368d895702 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:53.572255 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff18dbd-6698-41a3-8bb5-367631d50b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:53.573148 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff18dbd-6698-41a3-8bb5-367631d50b8b None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:53.573438 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff18dbd-6698-41a3-8bb5-367631d50b8b None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:53.573721 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff18dbd-6698-41a3-8bb5-367631d50b8b None admin] PATH_INFO: `/v3/projects/2e8284c9d9284b72a1b442e53a240f9a` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:53.574915 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff18dbd-6698-41a3-8bb5-367631d50b8b None admin] RBAC: Authorizing `identity:get_project(project_id=2e8284c9d9284b72a1b442e53a240f9a)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:53.576354 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff18dbd-6698-41a3-8bb5-367631d50b8b None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:53.578097 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 595/1199] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 17 16:42:53 2022] GET /identity/v3/projects/2e8284c9d9284b72a1b442e53a240f9a => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:42:53.584413 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd08b28-55ed-4c7c-a108-7cb5b4482db1 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:42:53.647032 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:42:53.686499 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd08b28-55ed-4c7c-a108-7cb5b4482db1 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:42:53.722321 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9fbe5e-f31c-486d-abb4-e1e9ae0d73d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:42:53.722985 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fbe5e-f31c-486d-abb4-e1e9ae0d73d1 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:42:53.723270 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fbe5e-f31c-486d-abb4-e1e9ae0d73d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:42:53.723549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9fbe5e-f31c-486d-abb4-e1e9ae0d73d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:42:53.724388 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9fbe5e-f31c-486d-abb4-e1e9ae0d73d1 None admin] RBAC: Adding query filter params (role.id=1025051975f64f689effa58dc4259e36, scope.project.id=2e8284c9d9284b72a1b442e53a240f9a, user.id=c136c233a28c44ad987fdb0cda72bdac) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:42:53.724830 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9fbe5e-f31c-486d-abb4-e1e9ae0d73d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:42:53.726278 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9fbe5e-f31c-486d-abb4-e1e9ae0d73d1 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:42:53.732607 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 605/1200] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:42:53 2022] GET /identity/v3/role_assignments?user.id=c136c233a28c44ad987fdb0cda72bdac&scope.project.id=2e8284c9d9284b72a1b442e53a240f9a&role.id=1025051975f64f689effa58dc4259e36 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:43:03.566958 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=79 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:43:03.729843 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=37 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:10.513033 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8104888-c99a-4e64-be3c-643337cabed5 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:10.513661 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8104888-c99a-4e64-be3c-643337cabed5 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:10.513661 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8104888-c99a-4e64-be3c-643337cabed5 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:10.514909 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 596/1201] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:44:10.520295 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b4e83-5547-40b2-99eb-e5602e49e61b None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:10.520723 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b4e83-5547-40b2-99eb-e5602e49e61b None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:10.520908 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b4e83-5547-40b2-99eb-e5602e49e61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:10.538037 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-449b4e83-5547-40b2-99eb-e5602e49e61b None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:44:10.539268 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-449b4e83-5547-40b2-99eb-e5602e49e61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:44:10.609102 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 606/1202] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:44:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:44:10.634442 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8104888-c99a-4e64-be3c-643337cabed5 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:10.750229 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8104888-c99a-4e64-be3c-643337cabed5 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:10.789375 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0305d7f7-c95c-48cb-85a2-1e31a9804242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:10.790197 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0305d7f7-c95c-48cb-85a2-1e31a9804242 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:10.790483 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0305d7f7-c95c-48cb-85a2-1e31a9804242 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:10.790764 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0305d7f7-c95c-48cb-85a2-1e31a9804242 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:10.803780 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0305d7f7-c95c-48cb-85a2-1e31a9804242 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:10.808492 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0305d7f7-c95c-48cb-85a2-1e31a9804242 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:10.825892 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-0305d7f7-c95c-48cb-85a2-1e31a9804242 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:44:10.825892 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:10.825892 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:44:10.825892 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:44:10.825892 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:44:10.825892 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:44:10.825892 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:44:10.825892 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:44:10.825892 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:44:10.825892 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:44:10.825892 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:44:10.825892 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:44:10.825892 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:44:10.825892 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:44:10.825892 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:44:10.827635 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:44:10.827635 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:44:10.827635 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:44:10.827635 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:44:10.827635 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:44:10.827635 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:44:10.827635 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:44:10.827635 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:44:10.827635 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:44:10.827635 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:44:10.827635 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:44:10.827635 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:44:10.827635 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:44:10.827635 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:44:10.827635 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:44:10.827635 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:44:10.827635 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:44:10.831612 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:44:10.831612 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:44:10.831612 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:44:10.831612 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:44:10.831612 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 597/1203] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:44:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:44:10.833923 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b4e83-5547-40b2-99eb-e5602e49e61b None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:10.946007 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b4e83-5547-40b2-99eb-e5602e49e61b None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:10.978359 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:10.984365 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e0f09-1422-4ce1-b43b-42d52efd5207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:10.985188 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e0f09-1422-4ce1-b43b-42d52efd5207 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:10.985471 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e0f09-1422-4ce1-b43b-42d52efd5207 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:10.985749 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e0f09-1422-4ce1-b43b-42d52efd5207 None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:10.986666 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e0f09-1422-4ce1-b43b-42d52efd5207 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:10.986960 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e0f09-1422-4ce1-b43b-42d52efd5207 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:10.988508 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e0f09-1422-4ce1-b43b-42d52efd5207 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:11.000564 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 607/1204] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:44:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:44:11.009487 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0305d7f7-c95c-48cb-85a2-1e31a9804242 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:11.063423 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:11.111070 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0305d7f7-c95c-48cb-85a2-1e31a9804242 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:11.147049 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597eb18-dd63-4322-aaed-c9a88d8c95d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:11.147723 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597eb18-dd63-4322-aaed-c9a88d8c95d9 None admin] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:11.148014 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597eb18-dd63-4322-aaed-c9a88d8c95d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:11.148296 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597eb18-dd63-4322-aaed-c9a88d8c95d9 None admin] PATH_INFO: `/v3/users` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:11.149039 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597eb18-dd63-4322-aaed-c9a88d8c95d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:11.150694 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597eb18-dd63-4322-aaed-c9a88d8c95d9 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:11.277990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 598/1205] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 17 16:44:11 2022] POST /identity/v3/users => generated 251 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:44:13.341201 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462d36c-d825-4a29-845a-05bc209674bc None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:13.341772 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462d36c-d825-4a29-845a-05bc209674bc None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:13.341772 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462d36c-d825-4a29-845a-05bc209674bc None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:13.343147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 608/1206] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:44:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:44:13.349369 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe89c6a-7d93-4849-abd9-1c4089f2cd6c None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:13.349826 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe89c6a-7d93-4849-abd9-1c4089f2cd6c None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:13.349923 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe89c6a-7d93-4849-abd9-1c4089f2cd6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:13.367869 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-dbe89c6a-7d93-4849-abd9-1c4089f2cd6c None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:44:13.369103 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-dbe89c6a-7d93-4849-abd9-1c4089f2cd6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:44:13.439353 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 599/1207] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:44:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:44:13.463799 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b462d36c-d825-4a29-845a-05bc209674bc None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:13.570438 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b462d36c-d825-4a29-845a-05bc209674bc None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:13.606221 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a8ce7d-aa4a-4a10-83b0-93506a601c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:13.607019 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8ce7d-aa4a-4a10-83b0-93506a601c50 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:13.607304 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8ce7d-aa4a-4a10-83b0-93506a601c50 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:13.607585 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a8ce7d-aa4a-4a10-83b0-93506a601c50 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:13.618046 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a8ce7d-aa4a-4a10-83b0-93506a601c50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:44:13.625751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-d8a8ce7d-aa4a-4a10-83b0-93506a601c50 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:44:13.625751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:13.625751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:44:13.625751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:44:13.625751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:44:13.625751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:44:13.625751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:44:13.625751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:44:13.625751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:44:13.625751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:44:13.625751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:44:13.625751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:44:13.625751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:44:13.625751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:44:13.625751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:44:13.625751 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:44:13.632582 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:44:13.632582 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:44:13.632582 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:44:13.632582 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:44:13.632582 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:44:13.632582 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:44:13.632582 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:44:13.632582 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:44:13.632582 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:44:13.632582 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:44:13.632582 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:44:13.632582 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:44:13.632582 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:44:13.632582 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:44:13.632582 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:44:13.632582 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:44:13.633628 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:44:13.633628 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:44:13.633628 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:44:13.633628 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:44:13.633628 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:44:13.633628 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:44:13.633628 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:44:13.633628 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:44:13.633628 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:44:13.633628 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:44:13.633628 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:44:13.633628 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:44:13.633628 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 609/1208] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 17 16:44:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:44:13.634704 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe89c6a-7d93-4849-abd9-1c4089f2cd6c None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:13.677971 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:13.741088 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe89c6a-7d93-4849-abd9-1c4089f2cd6c None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:13.775772 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818eeefc-4653-4b66-b7c6-9a88cf27b45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:13.776451 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818eeefc-4653-4b66-b7c6-9a88cf27b45c None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:13.776736 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818eeefc-4653-4b66-b7c6-9a88cf27b45c None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:13.777047 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818eeefc-4653-4b66-b7c6-9a88cf27b45c None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:13.777796 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818eeefc-4653-4b66-b7c6-9a88cf27b45c None admin] RBAC: Adding query filter params (name=service) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:13.778190 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818eeefc-4653-4b66-b7c6-9a88cf27b45c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:13.779558 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818eeefc-4653-4b66-b7c6-9a88cf27b45c None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:13.789154 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 600/1209] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 17 16:44:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:44:13.795057 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a8ce7d-aa4a-4a10-83b0-93506a601c50 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:13.848155 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:13.897081 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a8ce7d-aa4a-4a10-83b0-93506a601c50 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:13.931282 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48d682-6385-46f1-b2c9-13268cf26fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:13.932082 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48d682-6385-46f1-b2c9-13268cf26fc1 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:13.932370 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48d682-6385-46f1-b2c9-13268cf26fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:13.932651 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48d682-6385-46f1-b2c9-13268cf26fc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:13.944080 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48d682-6385-46f1-b2c9-13268cf26fc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:13.945561 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48d682-6385-46f1-b2c9-13268cf26fc1 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:13.963128 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-ec48d682-6385-46f1-b2c9-13268cf26fc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:44:13.963128 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:13.963128 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:44:13.963128 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:44:13.963128 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:44:13.963128 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:44:13.963128 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:44:13.963128 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:44:13.963128 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:44:13.963128 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:44:13.963128 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:44:13.963128 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:44:13.963128 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:44:13.963128 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:44:13.963128 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:44:13.964235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:44:13.964235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:44:13.964235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:44:13.964235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:44:13.964235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:44:13.964235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:44:13.964235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:44:13.964235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:44:13.964235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:44:13.964235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:44:13.964235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:44:13.964235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:44:13.964235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:44:13.964235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:44:13.964235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:44:13.964235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:44:13.964235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:44:13.965588 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:44:13.965588 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:44:13.965588 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:44:13.965588 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:44:13.965588 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 610/1210] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:44:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:44:13.970179 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818eeefc-4653-4b66-b7c6-9a88cf27b45c None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:14.076420 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:14.079610 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818eeefc-4653-4b66-b7c6-9a88cf27b45c None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:14.115888 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8071b-bee0-4984-a0e5-bf1181339515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:14.116579 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8071b-bee0-4984-a0e5-bf1181339515 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:14.116895 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8071b-bee0-4984-a0e5-bf1181339515 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:14.117179 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8071b-bee0-4984-a0e5-bf1181339515 None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:14.117929 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b8071b-bee0-4984-a0e5-bf1181339515 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:14.118237 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b8071b-bee0-4984-a0e5-bf1181339515 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:14.119667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b8071b-bee0-4984-a0e5-bf1181339515 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:14.136199 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 601/1211] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:44:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:44:14.141207 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48d682-6385-46f1-b2c9-13268cf26fc1 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:14.248796 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48d682-6385-46f1-b2c9-13268cf26fc1 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:14.250375 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:14.283687 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333cdb1-3fdc-4352-be17-e73356c36269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:14.284479 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333cdb1-3fdc-4352-be17-e73356c36269 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:14.284868 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333cdb1-3fdc-4352-be17-e73356c36269 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:14.285323 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7333cdb1-3fdc-4352-be17-e73356c36269 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:14.292681 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333cdb1-3fdc-4352-be17-e73356c36269 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:14.294153 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7333cdb1-3fdc-4352-be17-e73356c36269 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:14.309646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-7333cdb1-3fdc-4352-be17-e73356c36269 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 17 16:44:14.309646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:14.309646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:44:14.309646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:44:14.309646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:44:14.309646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:44:14.309646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:44:14.309646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:44:14.309646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:44:14.309646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:44:14.309646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:44:14.309646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:44:14.309646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:44:14.309646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:44:14.309646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:44:14.309646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:44:14.311087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:44:14.311087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:44:14.311087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:44:14.311087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:44:14.311087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:44:14.311087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:44:14.311087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:44:14.311087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:44:14.311087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:44:14.311087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:44:14.311087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:44:14.311087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:44:14.311087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:44:14.311087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:44:14.311087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:44:14.311087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:44:14.311087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:44:14.311087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:44:14.312045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:44:14.312045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:44:14.312045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:44:14.312045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:44:14.312045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:44:14.312045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:44:14.312045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:44:14.312045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:44:14.312045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 17 16:44:14.312045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:44:14.312045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 611/1212] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:44:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:44:14.317872 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8071b-bee0-4984-a0e5-bf1181339515 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:14.424830 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8071b-bee0-4984-a0e5-bf1181339515 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:14.460215 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfd84a-71ae-48fe-96ec-6281990a9002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:14.460927 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfd84a-71ae-48fe-96ec-6281990a9002 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:14.461217 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfd84a-71ae-48fe-96ec-6281990a9002 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:14.461499 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cfd84a-71ae-48fe-96ec-6281990a9002 None admin] PATH_INFO: `/v3/users` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:14.462344 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cfd84a-71ae-48fe-96ec-6281990a9002 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:14.462657 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cfd84a-71ae-48fe-96ec-6281990a9002 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:14.464094 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cfd84a-71ae-48fe-96ec-6281990a9002 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:14.481653 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 602/1213] 10.222.0.40 () {56 vars in 1143 bytes} [Sun Jul 17 16:44:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:44:14.487650 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333cdb1-3fdc-4352-be17-e73356c36269 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:14.597081 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7333cdb1-3fdc-4352-be17-e73356c36269 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:14.635392 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d591e-6e0d-4788-aae3-3209a5d2815e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:14.635392 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d591e-6e0d-4788-aae3-3209a5d2815e None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:14.635392 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d591e-6e0d-4788-aae3-3209a5d2815e None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:14.635392 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d591e-6e0d-4788-aae3-3209a5d2815e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:14.664755 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:14.702695 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354d591e-6e0d-4788-aae3-3209a5d2815e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:14.704093 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354d591e-6e0d-4788-aae3-3209a5d2815e None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:14.799777 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 612/1214] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:44:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 17 16:44:14.808272 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfd84a-71ae-48fe-96ec-6281990a9002 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:14.828337 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:14.910429 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cfd84a-71ae-48fe-96ec-6281990a9002 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:14.945994 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09839e5-d2c7-455b-b26b-5630d329618d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:14.946828 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09839e5-d2c7-455b-b26b-5630d329618d None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:14.947132 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09839e5-d2c7-455b-b26b-5630d329618d None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:14.947397 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09839e5-d2c7-455b-b26b-5630d329618d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:14.958984 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09839e5-d2c7-455b-b26b-5630d329618d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:14.960436 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09839e5-d2c7-455b-b26b-5630d329618d None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:14.978602 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-e09839e5-d2c7-455b-b26b-5630d329618d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:44:14.978602 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:14.978602 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:44:14.978602 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:44:14.978602 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:44:14.978602 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:44:14.978602 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:44:14.978602 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:44:14.978602 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:44:14.978602 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:44:14.978602 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:44:14.978602 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:44:14.978602 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:44:14.978602 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:44:14.978602 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:44:14.982146 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:44:14.982146 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:44:14.982146 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:44:14.982146 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:44:14.982146 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:44:14.982146 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:44:14.982146 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:44:14.982146 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:44:14.982146 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:44:14.982146 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:44:14.982146 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:44:14.982146 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:44:14.982146 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:44:14.982146 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:44:14.982146 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:44:14.982146 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:44:14.982146 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:44:14.983116 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:44:14.983116 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:44:14.983116 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:44:14.983116 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:44:14.983116 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 603/1215] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:44:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:44:14.985939 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d591e-6e0d-4788-aae3-3209a5d2815e None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:15.049963 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:15.095526 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d591e-6e0d-4788-aae3-3209a5d2815e None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:15.131718 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b4784-b54b-41cf-94fd-f1e3833b6f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:15.132434 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b4784-b54b-41cf-94fd-f1e3833b6f5a None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:15.132725 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b4784-b54b-41cf-94fd-f1e3833b6f5a None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:15.133040 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b4784-b54b-41cf-94fd-f1e3833b6f5a None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:15.134520 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b4784-b54b-41cf-94fd-f1e3833b6f5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:15.134520 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b4784-b54b-41cf-94fd-f1e3833b6f5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:15.135512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b4784-b54b-41cf-94fd-f1e3833b6f5a None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:15.148079 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 613/1216] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:44:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:44:15.154072 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09839e5-d2c7-455b-b26b-5630d329618d None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:15.232254 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:15.264847 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09839e5-d2c7-455b-b26b-5630d329618d None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:15.297666 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c87efb-d9ce-4e19-b6f1-65bd48957c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:15.298540 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c87efb-d9ce-4e19-b6f1-65bd48957c3c None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:15.298824 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c87efb-d9ce-4e19-b6f1-65bd48957c3c None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:15.299101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c87efb-d9ce-4e19-b6f1-65bd48957c3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:15.306726 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c87efb-d9ce-4e19-b6f1-65bd48957c3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:15.308172 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c87efb-d9ce-4e19-b6f1-65bd48957c3c None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:15.323494 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-67c87efb-d9ce-4e19-b6f1-65bd48957c3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:44:15.323494 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:15.323494 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:44:15.323494 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:44:15.323494 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:44:15.323494 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:44:15.323494 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:44:15.323494 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:44:15.323494 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:44:15.323494 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:44:15.323494 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:44:15.323494 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:44:15.323494 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:44:15.323494 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:44:15.323494 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:44:15.324629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:44:15.324629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:44:15.324629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:44:15.324629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:44:15.324629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:44:15.324629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:44:15.324629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:44:15.324629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:44:15.324629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:44:15.324629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:44:15.324629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:44:15.324629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:44:15.324629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:44:15.324629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:44:15.324629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:44:15.324629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:44:15.324629 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:44:15.325726 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:44:15.325726 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:44:15.325726 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:44:15.325726 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:44:15.325726 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:44:15.325726 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:44:15.325726 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:44:15.325726 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:44:15.325726 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 604/1217] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:44:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:44:15.330808 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b4784-b54b-41cf-94fd-f1e3833b6f5a None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:15.432514 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b4784-b54b-41cf-94fd-f1e3833b6f5a None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:15.455716 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:15.467737 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9004dc4-6ca9-4508-9f6f-f741f708735f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:15.468443 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9004dc4-6ca9-4508-9f6f-f741f708735f None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:15.468755 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9004dc4-6ca9-4508-9f6f-f741f708735f None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:15.469060 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9004dc4-6ca9-4508-9f6f-f741f708735f None admin] PATH_INFO: `/v3/projects` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:15.469926 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9004dc4-6ca9-4508-9f6f-f741f708735f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:15.470238 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9004dc4-6ca9-4508-9f6f-f741f708735f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:15.471670 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9004dc4-6ca9-4508-9f6f-f741f708735f None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:15.483783 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 614/1218] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:44:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:44:15.490898 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c87efb-d9ce-4e19-b6f1-65bd48957c3c None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:15.593712 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c87efb-d9ce-4e19-b6f1-65bd48957c3c None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:15.623430 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:15.629890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b560d87-9045-42f9-b37c-6bbf2d8416ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:15.630554 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b560d87-9045-42f9-b37c-6bbf2d8416ff None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:15.630843 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b560d87-9045-42f9-b37c-6bbf2d8416ff None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:15.631125 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b560d87-9045-42f9-b37c-6bbf2d8416ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:15.631967 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b560d87-9045-42f9-b37c-6bbf2d8416ff None admin] RBAC: Adding query filter params (role.id=adb9c27c6e5246fdb0cf386358a822cd, scope.project.id=927057f9a5af4833aeac9992203a223c, user.id=8e332b0ae0f64e089a01c91df2d9f28f) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:15.632342 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b560d87-9045-42f9-b37c-6bbf2d8416ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:15.633804 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b560d87-9045-42f9-b37c-6bbf2d8416ff None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:15.639392 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 605/1219] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:44:15 2022] GET /identity/v3/role_assignments?user.id=8e332b0ae0f64e089a01c91df2d9f28f&scope.project.id=927057f9a5af4833aeac9992203a223c&role.id=adb9c27c6e5246fdb0cf386358a822cd => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:44:17.738424 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b511c1b-eb31-4c87-a37a-67a8f899764a None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:17.738986 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b511c1b-eb31-4c87-a37a-67a8f899764a None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:17.739099 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b511c1b-eb31-4c87-a37a-67a8f899764a None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:17.740727 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 615/1220] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:44:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:44:17.746180 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9399d8-b46a-4bf8-9eba-fba7a049ced2 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:17.746646 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9399d8-b46a-4bf8-9eba-fba7a049ced2 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:17.746800 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9399d8-b46a-4bf8-9eba-fba7a049ced2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:17.764907 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-1d9399d8-b46a-4bf8-9eba-fba7a049ced2 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:44:17.768246 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-1d9399d8-b46a-4bf8-9eba-fba7a049ced2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:44:17.842165 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 606/1221] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:44:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 17 16:44:17.865436 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b511c1b-eb31-4c87-a37a-67a8f899764a None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:17.974558 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b511c1b-eb31-4c87-a37a-67a8f899764a None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:18.011842 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aef3dc-404d-45c7-ab69-2627cb2e779a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:18.012575 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aef3dc-404d-45c7-ab69-2627cb2e779a None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:18.012892 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aef3dc-404d-45c7-ab69-2627cb2e779a None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:18.013178 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aef3dc-404d-45c7-ab69-2627cb2e779a None admin] PATH_INFO: `/v3/roles/service` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:18.024500 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aef3dc-404d-45c7-ab69-2627cb2e779a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:44:18.032375 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-92aef3dc-404d-45c7-ab69-2627cb2e779a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:44:18.032375 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:18.032375 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:44:18.032375 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:44:18.032375 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:44:18.032375 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:44:18.032375 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:44:18.032375 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:44:18.032375 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:44:18.032375 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:44:18.032375 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:44:18.032375 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:44:18.032375 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:44:18.032375 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:44:18.032375 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:44:18.032375 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:44:18.033549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:44:18.033549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:44:18.033549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:44:18.033549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:44:18.033549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:44:18.033549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:44:18.033549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:44:18.033549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:44:18.033549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:44:18.033549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:44:18.033549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:44:18.033549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:44:18.033549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:44:18.033549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:44:18.033549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:44:18.033549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:44:18.034517 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:44:18.034517 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:44:18.034517 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:44:18.034517 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:44:18.034517 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:44:18.034517 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:44:18.034517 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:44:18.034517 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:44:18.034517 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:44:18.034517 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:44:18.034517 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:44:18.034517 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:44:18.034517 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 616/1222] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 17 16:44:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:44:18.039987 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9399d8-b46a-4bf8-9eba-fba7a049ced2 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:18.145404 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9399d8-b46a-4bf8-9eba-fba7a049ced2 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:18.171790 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:18.183595 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802da8f0-0373-4df3-93e6-6233c089fe5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:18.184282 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802da8f0-0373-4df3-93e6-6233c089fe5e None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:18.184573 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802da8f0-0373-4df3-93e6-6233c089fe5e None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:18.184888 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802da8f0-0373-4df3-93e6-6233c089fe5e None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:18.185910 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802da8f0-0373-4df3-93e6-6233c089fe5e None admin] RBAC: Adding query filter params (name=service) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:18.186271 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802da8f0-0373-4df3-93e6-6233c089fe5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:18.187664 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802da8f0-0373-4df3-93e6-6233c089fe5e None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:18.197183 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 607/1223] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 17 16:44:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:44:18.203387 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aef3dc-404d-45c7-ab69-2627cb2e779a None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:18.248342 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:18.306105 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aef3dc-404d-45c7-ab69-2627cb2e779a None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:18.343310 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7345273-17ae-4b2e-9336-d7261cbe3773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:18.344126 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7345273-17ae-4b2e-9336-d7261cbe3773 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:18.344416 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7345273-17ae-4b2e-9336-d7261cbe3773 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:18.344689 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7345273-17ae-4b2e-9336-d7261cbe3773 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:18.356111 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7345273-17ae-4b2e-9336-d7261cbe3773 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:18.357655 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7345273-17ae-4b2e-9336-d7261cbe3773 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:18.375478 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-c7345273-17ae-4b2e-9336-d7261cbe3773 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:44:18.375478 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:18.375478 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:44:18.375478 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:44:18.375478 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:44:18.375478 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:44:18.375478 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:44:18.375478 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:44:18.375478 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:44:18.375478 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:44:18.375478 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:44:18.375478 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:44:18.375478 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:44:18.375478 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:44:18.375478 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:44:18.376687 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:44:18.376687 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:44:18.376687 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:44:18.376687 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:44:18.376687 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:44:18.376687 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:44:18.376687 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:44:18.376687 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:44:18.376687 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:44:18.376687 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:44:18.376687 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:44:18.376687 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:44:18.376687 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:44:18.376687 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:44:18.376687 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:44:18.376687 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:44:18.376687 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:44:18.387142 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:44:18.387142 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:44:18.387142 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:44:18.387142 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:44:18.387142 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 617/1224] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:44:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:44:18.387499 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802da8f0-0373-4df3-93e6-6233c089fe5e None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:18.504193 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802da8f0-0373-4df3-93e6-6233c089fe5e None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:18.539397 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57608c55-b5bb-46bd-800c-ec6bbb029701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:18.540088 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57608c55-b5bb-46bd-800c-ec6bbb029701 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:18.540373 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57608c55-b5bb-46bd-800c-ec6bbb029701 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:18.540806 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57608c55-b5bb-46bd-800c-ec6bbb029701 None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:18.545782 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57608c55-b5bb-46bd-800c-ec6bbb029701 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:18.545782 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57608c55-b5bb-46bd-800c-ec6bbb029701 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:18.545782 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57608c55-b5bb-46bd-800c-ec6bbb029701 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:18.557314 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 608/1225] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:44:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:44:18.563301 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7345273-17ae-4b2e-9336-d7261cbe3773 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:18.664282 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:18.670860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7345273-17ae-4b2e-9336-d7261cbe3773 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:18.707968 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8650e7-6ad5-4ca6-b4b9-18893d94cf09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:18.708833 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8650e7-6ad5-4ca6-b4b9-18893d94cf09 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:18.709124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8650e7-6ad5-4ca6-b4b9-18893d94cf09 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:18.709403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8650e7-6ad5-4ca6-b4b9-18893d94cf09 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:18.717227 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8650e7-6ad5-4ca6-b4b9-18893d94cf09 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:18.718728 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8650e7-6ad5-4ca6-b4b9-18893d94cf09 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:18.734011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-bf8650e7-6ad5-4ca6-b4b9-18893d94cf09 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 17 16:44:18.734011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:18.734011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:44:18.734011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:44:18.734011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:44:18.734011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:44:18.734011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:44:18.734011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:44:18.734011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:44:18.734011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:44:18.734011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:44:18.734011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:44:18.734011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:44:18.734011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:44:18.734011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:44:18.734011 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:44:18.735197 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:44:18.735197 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:44:18.735197 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:44:18.735197 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:44:18.735197 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:44:18.735197 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:44:18.735197 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:44:18.735197 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:44:18.735197 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:44:18.735197 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:44:18.735197 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:44:18.735197 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:44:18.735197 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:44:18.735197 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:44:18.735197 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:44:18.735197 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:44:18.735197 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:44:18.735197 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:44:18.736329 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:44:18.736329 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:44:18.736329 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:44:18.736329 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:44:18.736329 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:44:18.736329 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:44:18.736329 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:44:18.736329 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:44:18.736329 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 17 16:44:18.736329 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:44:18.736329 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 618/1226] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:44:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:44:18.741983 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57608c55-b5bb-46bd-800c-ec6bbb029701 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:18.797526 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:18.854169 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57608c55-b5bb-46bd-800c-ec6bbb029701 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:18.889863 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758b40b-09f3-47ca-a859-5c2f50010fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:18.890541 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758b40b-09f3-47ca-a859-5c2f50010fc8 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:18.890828 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758b40b-09f3-47ca-a859-5c2f50010fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:18.891107 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758b40b-09f3-47ca-a859-5c2f50010fc8 None admin] PATH_INFO: `/v3/users` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:18.891950 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1758b40b-09f3-47ca-a859-5c2f50010fc8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:18.892261 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1758b40b-09f3-47ca-a859-5c2f50010fc8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:18.893731 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1758b40b-09f3-47ca-a859-5c2f50010fc8 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:18.917032 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 609/1227] 10.222.0.40 () {56 vars in 1143 bytes} [Sun Jul 17 16:44:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:44:18.918278 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8650e7-6ad5-4ca6-b4b9-18893d94cf09 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:19.023431 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8650e7-6ad5-4ca6-b4b9-18893d94cf09 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:19.063130 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40211ecf-1849-4c01-9df2-16ba9ef0ae63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:19.063936 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40211ecf-1849-4c01-9df2-16ba9ef0ae63 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:19.064225 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40211ecf-1849-4c01-9df2-16ba9ef0ae63 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:19.064503 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40211ecf-1849-4c01-9df2-16ba9ef0ae63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:19.076300 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40211ecf-1849-4c01-9df2-16ba9ef0ae63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:19.077857 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40211ecf-1849-4c01-9df2-16ba9ef0ae63 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:19.095634 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-40211ecf-1849-4c01-9df2-16ba9ef0ae63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:44:19.095634 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:19.095634 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:44:19.095634 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:44:19.095634 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:44:19.095634 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:44:19.095634 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:44:19.095634 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:44:19.095634 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:44:19.095634 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:44:19.095634 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:44:19.095634 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:44:19.095634 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:44:19.095634 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:44:19.095634 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:44:19.096755 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:44:19.096755 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:44:19.096755 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:44:19.096755 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:44:19.096755 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:44:19.096755 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:44:19.096755 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:44:19.096755 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:44:19.096755 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:44:19.096755 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:44:19.096755 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:44:19.096755 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:44:19.096755 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:44:19.096755 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:44:19.096755 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:44:19.096755 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:44:19.096755 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:44:19.098446 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:44:19.098446 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:44:19.098446 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:44:19.098446 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:44:19.098446 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 619/1228] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:44:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:44:19.103668 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758b40b-09f3-47ca-a859-5c2f50010fc8 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:19.209043 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758b40b-09f3-47ca-a859-5c2f50010fc8 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:19.211406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:19.244917 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b25a4d-fab9-4dcc-9cda-a7f67e4979be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:19.245598 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b25a4d-fab9-4dcc-9cda-a7f67e4979be None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:19.245888 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b25a4d-fab9-4dcc-9cda-a7f67e4979be None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:19.246168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b25a4d-fab9-4dcc-9cda-a7f67e4979be None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:19.246976 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b25a4d-fab9-4dcc-9cda-a7f67e4979be None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:19.247294 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b25a4d-fab9-4dcc-9cda-a7f67e4979be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:19.248724 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b25a4d-fab9-4dcc-9cda-a7f67e4979be None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:19.260616 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 610/1229] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:44:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:44:19.266823 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40211ecf-1849-4c01-9df2-16ba9ef0ae63 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:19.276043 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:19.367343 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40211ecf-1849-4c01-9df2-16ba9ef0ae63 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:19.400594 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf962be-2887-47d6-b070-861ececf3b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:19.401406 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf962be-2887-47d6-b070-861ececf3b66 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:19.401686 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf962be-2887-47d6-b070-861ececf3b66 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:19.401957 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf962be-2887-47d6-b070-861ececf3b66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:19.408491 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf962be-2887-47d6-b070-861ececf3b66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:19.410703 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf962be-2887-47d6-b070-861ececf3b66 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:19.426654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-bdf962be-2887-47d6-b070-861ececf3b66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:44:19.426654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:19.426654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:44:19.426654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:44:19.426654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:44:19.426654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:44:19.426654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:44:19.426654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:44:19.426654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:44:19.426654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:44:19.426654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:44:19.426654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:44:19.426654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:44:19.426654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:44:19.426654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:44:19.427856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:44:19.427856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:44:19.427856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:44:19.427856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:44:19.427856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:44:19.427856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:44:19.427856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:44:19.427856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:44:19.427856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:44:19.427856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:44:19.427856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:44:19.427856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:44:19.427856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:44:19.427856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:44:19.427856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:44:19.427856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:44:19.427856 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:44:19.429245 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:44:19.429245 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:44:19.429245 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:44:19.429245 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:44:19.429245 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:44:19.429245 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:44:19.429245 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:44:19.429245 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:44:19.429245 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 620/1230] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:44:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:44:19.434322 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b25a4d-fab9-4dcc-9cda-a7f67e4979be None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:19.539210 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b25a4d-fab9-4dcc-9cda-a7f67e4979be None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:19.573555 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc33d3bb-200a-4e18-acb8-d6fcef66f888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:19.574221 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33d3bb-200a-4e18-acb8-d6fcef66f888 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:19.574507 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33d3bb-200a-4e18-acb8-d6fcef66f888 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:19.574788 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33d3bb-200a-4e18-acb8-d6fcef66f888 None admin] PATH_INFO: `/v3/projects` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:19.575583 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc33d3bb-200a-4e18-acb8-d6fcef66f888 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:19.575926 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc33d3bb-200a-4e18-acb8-d6fcef66f888 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:19.577556 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc33d3bb-200a-4e18-acb8-d6fcef66f888 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:19.591899 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 611/1231] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:44:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:44:19.599297 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf962be-2887-47d6-b070-861ececf3b66 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:19.666242 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:19.701101 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf962be-2887-47d6-b070-861ececf3b66 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:19.734848 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93136d-b478-45b5-a34f-34841251fcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:19.735733 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93136d-b478-45b5-a34f-34841251fcee None admin] REQUEST_METHOD: `PUT` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:19.736019 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93136d-b478-45b5-a34f-34841251fcee None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:19.736293 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93136d-b478-45b5-a34f-34841251fcee None admin] PATH_INFO: `/v3/projects/927057f9a5af4833aeac9992203a223c/users/8e332b0ae0f64e089a01c91df2d9f28f/roles/adb9c27c6e5246fdb0cf386358a822cd` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:19.753611 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff93136d-b478-45b5-a34f-34841251fcee None admin] RBAC: Authorizing `identity:create_grant(project_id=927057f9a5af4833aeac9992203a223c, user_id=8e332b0ae0f64e089a01c91df2d9f28f, role_id=adb9c27c6e5246fdb0cf386358a822cd)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:19.755958 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff93136d-b478-45b5-a34f-34841251fcee None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:19.856804 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 621/1232] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 17 16:44:19 2022] PUT /identity/v3/projects/927057f9a5af4833aeac9992203a223c/users/8e332b0ae0f64e089a01c91df2d9f28f/roles/adb9c27c6e5246fdb0cf386358a822cd => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 17 16:44:21.906508 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83106e8-676e-4bf0-b9d5-fd18091b502c None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:21.907076 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83106e8-676e-4bf0-b9d5-fd18091b502c None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:21.907076 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83106e8-676e-4bf0-b9d5-fd18091b502c None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:21.908484 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 612/1233] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:44:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:44:21.914725 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3731e-12e1-4652-a207-d41b4a3c607e None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:21.915276 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3731e-12e1-4652-a207-d41b4a3c607e None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:21.915462 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3731e-12e1-4652-a207-d41b4a3c607e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:21.932227 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-1ec3731e-12e1-4652-a207-d41b4a3c607e None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:44:21.941430 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-1ec3731e-12e1-4652-a207-d41b4a3c607e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:44:22.003263 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 622/1234] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:44:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:44:22.028699 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83106e8-676e-4bf0-b9d5-fd18091b502c None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:22.055407 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:22.133638 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83106e8-676e-4bf0-b9d5-fd18091b502c None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:22.168538 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4700b5-dc9e-4cd2-95ea-5a0d33d239e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:22.169339 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4700b5-dc9e-4cd2-95ea-5a0d33d239e0 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:22.169621 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4700b5-dc9e-4cd2-95ea-5a0d33d239e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:22.169894 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4700b5-dc9e-4cd2-95ea-5a0d33d239e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:22.181271 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4700b5-dc9e-4cd2-95ea-5a0d33d239e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:44:22.189366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-8b4700b5-dc9e-4cd2-95ea-5a0d33d239e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:44:22.189366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:22.189366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:44:22.189366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:44:22.189366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:44:22.189366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:44:22.189366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:44:22.189366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:44:22.189366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:44:22.189366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:44:22.189366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:44:22.189366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:44:22.189366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:44:22.189366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:44:22.189366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:44:22.189366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:44:22.192922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:44:22.192922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:44:22.192922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:44:22.192922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:44:22.192922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:44:22.192922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:44:22.192922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:44:22.192922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:44:22.192922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:44:22.192922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:44:22.192922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:44:22.192922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:44:22.192922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:44:22.192922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:44:22.192922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:44:22.192922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:44:22.193996 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:44:22.193996 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:44:22.193996 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:44:22.193996 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:44:22.193996 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:44:22.193996 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:44:22.193996 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:44:22.193996 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:44:22.193996 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:44:22.193996 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:44:22.193996 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 17 16:44:22.193996 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:44:22.193996 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 613/1235] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 17 16:44:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:44:22.197404 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3731e-12e1-4652-a207-d41b4a3c607e None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:22.243965 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:22.298460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3731e-12e1-4652-a207-d41b4a3c607e None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:22.336150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d422286-a9cf-4397-8ebd-bacbcbd2fe07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:22.336859 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d422286-a9cf-4397-8ebd-bacbcbd2fe07 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:22.337148 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d422286-a9cf-4397-8ebd-bacbcbd2fe07 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:22.337428 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d422286-a9cf-4397-8ebd-bacbcbd2fe07 None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:22.338182 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d422286-a9cf-4397-8ebd-bacbcbd2fe07 None admin] RBAC: Adding query filter params (name=service) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:22.338493 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d422286-a9cf-4397-8ebd-bacbcbd2fe07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:22.339866 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d422286-a9cf-4397-8ebd-bacbcbd2fe07 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:22.348287 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 623/1236] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 17 16:44:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:44:22.354481 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4700b5-dc9e-4cd2-95ea-5a0d33d239e0 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:22.441953 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:22.457412 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4700b5-dc9e-4cd2-95ea-5a0d33d239e0 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:22.490791 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9901d6f-fc57-4dcd-92ec-281e1947c429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:22.491595 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9901d6f-fc57-4dcd-92ec-281e1947c429 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:22.491878 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9901d6f-fc57-4dcd-92ec-281e1947c429 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:22.492153 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9901d6f-fc57-4dcd-92ec-281e1947c429 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:22.502608 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9901d6f-fc57-4dcd-92ec-281e1947c429 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:22.504693 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9901d6f-fc57-4dcd-92ec-281e1947c429 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:22.523214 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-b9901d6f-fc57-4dcd-92ec-281e1947c429 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:44:22.523214 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:22.523214 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:44:22.523214 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:44:22.523214 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:44:22.523214 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:44:22.523214 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:44:22.523214 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:44:22.523214 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:44:22.523214 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:44:22.523214 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:44:22.523214 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:44:22.523214 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:44:22.523214 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:44:22.523214 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:44:22.524311 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:44:22.524311 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:44:22.524311 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:44:22.524311 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:44:22.524311 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:44:22.524311 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:44:22.524311 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:44:22.524311 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:44:22.524311 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:44:22.524311 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:44:22.524311 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:44:22.524311 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:44:22.524311 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:44:22.524311 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:44:22.524311 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:44:22.524311 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:44:22.524311 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:44:22.525631 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:44:22.525631 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:44:22.525631 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:44:22.525631 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:44:22.525631 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 614/1237] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:44:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:44:22.530392 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d422286-a9cf-4397-8ebd-bacbcbd2fe07 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:22.633897 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d422286-a9cf-4397-8ebd-bacbcbd2fe07 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:22.642822 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:22.667213 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc65a1-f60f-4c6c-b101-48e1b68b462f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:22.667886 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc65a1-f60f-4c6c-b101-48e1b68b462f None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:22.668168 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc65a1-f60f-4c6c-b101-48e1b68b462f None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:22.668445 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc65a1-f60f-4c6c-b101-48e1b68b462f None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:22.669194 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc65a1-f60f-4c6c-b101-48e1b68b462f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:22.669503 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc65a1-f60f-4c6c-b101-48e1b68b462f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:22.670884 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc65a1-f60f-4c6c-b101-48e1b68b462f None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:22.683357 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 624/1238] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:44:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:44:22.689590 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9901d6f-fc57-4dcd-92ec-281e1947c429 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:22.803990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9901d6f-fc57-4dcd-92ec-281e1947c429 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:22.842637 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614340a4-1f82-48fd-b89f-bbb5257be9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:22.843473 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614340a4-1f82-48fd-b89f-bbb5257be9b1 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:22.843755 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614340a4-1f82-48fd-b89f-bbb5257be9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:22.844035 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614340a4-1f82-48fd-b89f-bbb5257be9b1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:22.847932 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:22.853756 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614340a4-1f82-48fd-b89f-bbb5257be9b1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:22.855575 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614340a4-1f82-48fd-b89f-bbb5257be9b1 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:22.872719 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-614340a4-1f82-48fd-b89f-bbb5257be9b1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 17 16:44:22.872719 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:22.872719 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:44:22.872719 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:44:22.872719 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:44:22.872719 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:44:22.872719 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:44:22.872719 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:44:22.872719 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:44:22.872719 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:44:22.872719 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:44:22.872719 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:44:22.872719 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:44:22.872719 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:44:22.872719 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:44:22.872719 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:44:22.877461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:44:22.877461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:44:22.877461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:44:22.877461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:44:22.877461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:44:22.877461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:44:22.877461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:44:22.877461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:44:22.877461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:44:22.877461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:44:22.877461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:44:22.877461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:44:22.877461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:44:22.877461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:44:22.877461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:44:22.877461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:44:22.877461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:44:22.877461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:44:22.878664 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:44:22.878664 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:44:22.878664 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:44:22.878664 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:44:22.878664 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:44:22.878664 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:44:22.878664 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:44:22.878664 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:44:22.878664 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 17 16:44:22.878664 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:44:22.878664 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 615/1239] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:44:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:44:22.880701 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc65a1-f60f-4c6c-b101-48e1b68b462f None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:22.982400 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc65a1-f60f-4c6c-b101-48e1b68b462f None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:23.017587 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9471087-deae-47fa-a1bd-8fef6fc355db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:23.018258 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9471087-deae-47fa-a1bd-8fef6fc355db None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:23.018540 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9471087-deae-47fa-a1bd-8fef6fc355db None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:23.018814 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9471087-deae-47fa-a1bd-8fef6fc355db None admin] PATH_INFO: `/v3/users` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:23.020409 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9471087-deae-47fa-a1bd-8fef6fc355db None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:23.020714 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9471087-deae-47fa-a1bd-8fef6fc355db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:23.022174 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9471087-deae-47fa-a1bd-8fef6fc355db None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:23.040490 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 625/1240] 10.222.0.40 () {56 vars in 1143 bytes} [Sun Jul 17 16:44:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:44:23.046533 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614340a4-1f82-48fd-b89f-bbb5257be9b1 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:23.152496 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614340a4-1f82-48fd-b89f-bbb5257be9b1 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:23.189247 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd9109-d3a1-45d6-a141-35f12fd632e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:23.189904 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd9109-d3a1-45d6-a141-35f12fd632e2 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:23.190188 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd9109-d3a1-45d6-a141-35f12fd632e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:23.190467 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd9109-d3a1-45d6-a141-35f12fd632e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:23.256025 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:23.263538 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dd9109-d3a1-45d6-a141-35f12fd632e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:23.265121 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dd9109-d3a1-45d6-a141-35f12fd632e2 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:23.374092 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 616/1241] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:44:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 17 16:44:23.381196 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9471087-deae-47fa-a1bd-8fef6fc355db None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:23.411738 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:23.491864 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9471087-deae-47fa-a1bd-8fef6fc355db None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:23.528912 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb15b7a-d727-45cf-8ae6-f0cc34dbb034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:23.529725 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb15b7a-d727-45cf-8ae6-f0cc34dbb034 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:23.530008 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb15b7a-d727-45cf-8ae6-f0cc34dbb034 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:23.530282 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb15b7a-d727-45cf-8ae6-f0cc34dbb034 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:23.542425 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb15b7a-d727-45cf-8ae6-f0cc34dbb034 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:23.543952 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb15b7a-d727-45cf-8ae6-f0cc34dbb034 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:23.563422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-eeb15b7a-d727-45cf-8ae6-f0cc34dbb034 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:44:23.563422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:23.563422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:44:23.563422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:44:23.563422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:44:23.563422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:44:23.563422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:44:23.563422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:44:23.563422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:44:23.563422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:44:23.563422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:44:23.563422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:44:23.563422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:44:23.563422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:44:23.563422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:44:23.564514 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:44:23.564514 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:44:23.564514 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:44:23.564514 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:44:23.564514 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:44:23.564514 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:44:23.564514 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:44:23.564514 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:44:23.564514 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:44:23.564514 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:44:23.564514 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:44:23.564514 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:44:23.564514 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:44:23.564514 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:44:23.564514 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:44:23.564514 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:44:23.564514 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:44:23.565533 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:44:23.565533 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:44:23.565533 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:44:23.565533 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:44:23.565533 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 626/1242] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:44:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:44:23.571605 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd9109-d3a1-45d6-a141-35f12fd632e2 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:23.702486 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:23.709082 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd9109-d3a1-45d6-a141-35f12fd632e2 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:23.744474 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487bf83-73ea-4df4-b64a-4f3c3ba4a2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:23.745175 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487bf83-73ea-4df4-b64a-4f3c3ba4a2c4 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:23.745480 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487bf83-73ea-4df4-b64a-4f3c3ba4a2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:23.745743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487bf83-73ea-4df4-b64a-4f3c3ba4a2c4 None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:23.746483 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1487bf83-73ea-4df4-b64a-4f3c3ba4a2c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:23.746791 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1487bf83-73ea-4df4-b64a-4f3c3ba4a2c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:23.748219 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1487bf83-73ea-4df4-b64a-4f3c3ba4a2c4 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:23.760595 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 617/1243] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:44:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:44:23.766461 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb15b7a-d727-45cf-8ae6-f0cc34dbb034 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:23.858324 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:23.884652 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb15b7a-d727-45cf-8ae6-f0cc34dbb034 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:23.921355 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635a42d-3483-4198-93eb-178db394f4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:23.922166 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635a42d-3483-4198-93eb-178db394f4e5 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:23.922449 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635a42d-3483-4198-93eb-178db394f4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:23.922730 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635a42d-3483-4198-93eb-178db394f4e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:23.930568 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3635a42d-3483-4198-93eb-178db394f4e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:23.932019 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3635a42d-3483-4198-93eb-178db394f4e5 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:23.946972 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-3635a42d-3483-4198-93eb-178db394f4e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:44:23.946972 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:23.946972 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:44:23.946972 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:44:23.946972 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:44:23.946972 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:44:23.946972 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:44:23.946972 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:44:23.946972 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:44:23.946972 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:44:23.946972 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:44:23.946972 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:44:23.946972 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:44:23.946972 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:44:23.946972 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:44:23.949743 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:44:23.949743 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:44:23.949743 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:44:23.949743 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:44:23.949743 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:44:23.949743 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:44:23.949743 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:44:23.949743 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:44:23.949743 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:44:23.949743 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:44:23.949743 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:44:23.949743 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:44:23.949743 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:44:23.949743 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:44:23.949743 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:44:23.949743 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:44:23.949743 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:44:23.951233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:44:23.951233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:44:23.951233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:44:23.951233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:44:23.951233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:44:23.951233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:44:23.951233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:44:23.951233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:44:23.951233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 627/1244] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:44:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 17 16:44:23.954371 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487bf83-73ea-4df4-b64a-4f3c3ba4a2c4 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:24.064418 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487bf83-73ea-4df4-b64a-4f3c3ba4a2c4 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:24.101212 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf8efc3-b4e6-4c4b-873c-b8f35a494d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:24.101879 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8efc3-b4e6-4c4b-873c-b8f35a494d93 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:24.102161 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8efc3-b4e6-4c4b-873c-b8f35a494d93 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:24.102439 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8efc3-b4e6-4c4b-873c-b8f35a494d93 None admin] PATH_INFO: `/v3/projects` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:24.103232 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf8efc3-b4e6-4c4b-873c-b8f35a494d93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:24.103542 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf8efc3-b4e6-4c4b-873c-b8f35a494d93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:24.104991 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf8efc3-b4e6-4c4b-873c-b8f35a494d93 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:24.119474 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 618/1245] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:44:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:44:24.125732 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635a42d-3483-4198-93eb-178db394f4e5 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:24.238556 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3635a42d-3483-4198-93eb-178db394f4e5 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:24.275603 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6db01c-8b0b-4541-a495-6eaaeb99c46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:24.276265 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6db01c-8b0b-4541-a495-6eaaeb99c46e None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:24.276552 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6db01c-8b0b-4541-a495-6eaaeb99c46e None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:24.276858 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6db01c-8b0b-4541-a495-6eaaeb99c46e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:24.277699 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6db01c-8b0b-4541-a495-6eaaeb99c46e None admin] RBAC: Adding query filter params (role.id=adb9c27c6e5246fdb0cf386358a822cd, scope.project.id=927057f9a5af4833aeac9992203a223c, user.id=8e332b0ae0f64e089a01c91df2d9f28f) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:24.278007 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6db01c-8b0b-4541-a495-6eaaeb99c46e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:24.279451 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6db01c-8b0b-4541-a495-6eaaeb99c46e None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:24.285873 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 628/1246] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:44:24 2022] GET /identity/v3/role_assignments?user.id=8e332b0ae0f64e089a01c91df2d9f28f&scope.project.id=927057f9a5af4833aeac9992203a223c&role.id=adb9c27c6e5246fdb0cf386358a822cd => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:44:26.382311 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7541c7-be82-456c-8505-c6cfbe9a8ed7 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:26.382866 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7541c7-be82-456c-8505-c6cfbe9a8ed7 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:26.382866 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7541c7-be82-456c-8505-c6cfbe9a8ed7 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:26.384231 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 619/1247] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:44:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:44:26.389582 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5a4a8-abd7-401b-98d7-9331080c44dc None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:26.389986 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5a4a8-abd7-401b-98d7-9331080c44dc None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:26.394501 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5a4a8-abd7-401b-98d7-9331080c44dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:26.395415 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:26.407825 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-56f5a4a8-abd7-401b-98d7-9331080c44dc None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:44:26.409172 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-56f5a4a8-abd7-401b-98d7-9331080c44dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:44:26.483902 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 629/1248] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:44:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:44:26.511943 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7541c7-be82-456c-8505-c6cfbe9a8ed7 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:26.530735 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:26.618323 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7541c7-be82-456c-8505-c6cfbe9a8ed7 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:26.652370 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d33a7-929e-4e29-b501-5104156ca956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:26.653198 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d33a7-929e-4e29-b501-5104156ca956 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:26.653484 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d33a7-929e-4e29-b501-5104156ca956 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:26.653791 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d33a7-929e-4e29-b501-5104156ca956 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:26.664446 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7d33a7-929e-4e29-b501-5104156ca956 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:44:26.672360 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-ae7d33a7-929e-4e29-b501-5104156ca956 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:44:26.672360 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:26.672360 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:44:26.672360 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:44:26.672360 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:44:26.672360 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:44:26.672360 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:44:26.672360 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:44:26.672360 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:44:26.672360 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:44:26.672360 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:44:26.672360 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:44:26.672360 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:44:26.672360 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:44:26.672360 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:44:26.672360 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:44:26.673468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:44:26.673468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:44:26.673468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:44:26.673468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:44:26.673468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:44:26.673468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:44:26.673468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:44:26.673468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:44:26.673468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:44:26.673468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:44:26.673468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:44:26.673468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:44:26.673468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:44:26.673468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:44:26.673468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:44:26.673468 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:44:26.674518 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:44:26.674518 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:44:26.674518 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:44:26.674518 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:44:26.674518 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:44:26.674518 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:44:26.674518 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:44:26.674518 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:44:26.674518 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:44:26.674518 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:44:26.674518 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:44:26.674518 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:44:26.674518 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 620/1249] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:44:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:44:26.680253 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f5a4a8-abd7-401b-98d7-9331080c44dc None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:26.780865 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f5a4a8-abd7-401b-98d7-9331080c44dc None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:26.811472 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:26.817314 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd7fa30-8b69-4988-be29-cabd8d89c388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:26.817992 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7fa30-8b69-4988-be29-cabd8d89c388 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:26.818277 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7fa30-8b69-4988-be29-cabd8d89c388 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:26.818555 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7fa30-8b69-4988-be29-cabd8d89c388 None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:26.819301 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd7fa30-8b69-4988-be29-cabd8d89c388 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:26.819608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd7fa30-8b69-4988-be29-cabd8d89c388 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:26.823672 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd7fa30-8b69-4988-be29-cabd8d89c388 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:26.830671 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 630/1250] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:44:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:44:26.836431 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d33a7-929e-4e29-b501-5104156ca956 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:26.915110 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:26.938648 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d33a7-929e-4e29-b501-5104156ca956 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:26.972007 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae05c4b-3a92-4d81-9b60-18068e76fdd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:26.973537 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae05c4b-3a92-4d81-9b60-18068e76fdd6 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:26.973829 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae05c4b-3a92-4d81-9b60-18068e76fdd6 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:26.974175 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae05c4b-3a92-4d81-9b60-18068e76fdd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:26.984649 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae05c4b-3a92-4d81-9b60-18068e76fdd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:26.986182 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae05c4b-3a92-4d81-9b60-18068e76fdd6 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:27.003751 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-7ae05c4b-3a92-4d81-9b60-18068e76fdd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:44:27.003751 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:27.003751 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:44:27.003751 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:44:27.003751 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:44:27.003751 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:44:27.003751 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:44:27.003751 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:44:27.003751 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:44:27.003751 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:44:27.003751 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:44:27.003751 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:44:27.003751 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:44:27.003751 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:44:27.003751 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:44:27.009838 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:44:27.009838 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:44:27.009838 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:44:27.009838 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:44:27.009838 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:44:27.009838 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:44:27.009838 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:44:27.009838 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:44:27.009838 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:44:27.009838 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:44:27.009838 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:44:27.009838 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:44:27.009838 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:44:27.009838 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:44:27.009838 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:44:27.009838 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:44:27.009838 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:44:27.010991 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:44:27.010991 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:44:27.010991 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:44:27.010991 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:44:27.010991 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 621/1251] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:44:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:44:27.011370 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd7fa30-8b69-4988-be29-cabd8d89c388 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:27.114543 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd7fa30-8b69-4988-be29-cabd8d89c388 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:27.148382 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98098c0f-92d9-4b05-a56d-a1ad8c1a6231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:27.149069 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98098c0f-92d9-4b05-a56d-a1ad8c1a6231 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:27.149407 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98098c0f-92d9-4b05-a56d-a1ad8c1a6231 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:27.149683 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98098c0f-92d9-4b05-a56d-a1ad8c1a6231 None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:27.150425 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98098c0f-92d9-4b05-a56d-a1ad8c1a6231 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:27.150729 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98098c0f-92d9-4b05-a56d-a1ad8c1a6231 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:27.152093 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98098c0f-92d9-4b05-a56d-a1ad8c1a6231 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:27.164701 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 631/1252] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:44:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 17 16:44:27.171186 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae05c4b-3a92-4d81-9b60-18068e76fdd6 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:27.273335 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae05c4b-3a92-4d81-9b60-18068e76fdd6 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:27.300191 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:27.308241 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea953144-8229-42b8-b1c7-86ef364b9dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:27.309080 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea953144-8229-42b8-b1c7-86ef364b9dfa None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:27.309369 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea953144-8229-42b8-b1c7-86ef364b9dfa None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:27.309898 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea953144-8229-42b8-b1c7-86ef364b9dfa None admin] PATH_INFO: `/v3/users/placement` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:27.318393 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea953144-8229-42b8-b1c7-86ef364b9dfa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:27.319850 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea953144-8229-42b8-b1c7-86ef364b9dfa None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:27.334929 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-ea953144-8229-42b8-b1c7-86ef364b9dfa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 17 16:44:27.334929 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:27.334929 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:44:27.334929 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:44:27.334929 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:44:27.334929 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:44:27.334929 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:44:27.334929 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:44:27.334929 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:44:27.334929 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:44:27.334929 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:44:27.334929 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:44:27.334929 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:44:27.334929 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:44:27.334929 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:44:27.334929 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:44:27.336017 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:44:27.336017 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:44:27.336017 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:44:27.336017 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:44:27.336017 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:44:27.336017 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:44:27.336017 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:44:27.336017 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:44:27.336017 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:44:27.336017 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:44:27.336017 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:44:27.336017 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:44:27.336017 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:44:27.336017 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:44:27.336017 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:44:27.336017 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:44:27.336017 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:44:27.336017 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:44:27.337206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:44:27.337206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:44:27.337206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:44:27.337206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:44:27.337206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:44:27.337206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:44:27.337206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:44:27.337206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:44:27.337206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 17 16:44:27.337206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:44:27.337206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 622/1253] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:44:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:44:27.342124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98098c0f-92d9-4b05-a56d-a1ad8c1a6231 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:27.400775 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:27.449685 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98098c0f-92d9-4b05-a56d-a1ad8c1a6231 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:27.486103 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3962fc0-2049-489a-9ef8-6b866ddc4e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:27.486843 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3962fc0-2049-489a-9ef8-6b866ddc4e90 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:27.487136 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3962fc0-2049-489a-9ef8-6b866ddc4e90 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:27.487455 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3962fc0-2049-489a-9ef8-6b866ddc4e90 None admin] PATH_INFO: `/v3/users` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:27.488584 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3962fc0-2049-489a-9ef8-6b866ddc4e90 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:27.488935 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3962fc0-2049-489a-9ef8-6b866ddc4e90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:27.493264 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3962fc0-2049-489a-9ef8-6b866ddc4e90 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:27.510529 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 632/1254] 10.222.0.40 () {56 vars in 1143 bytes} [Sun Jul 17 16:44:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:44:27.516674 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea953144-8229-42b8-b1c7-86ef364b9dfa None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:27.631177 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea953144-8229-42b8-b1c7-86ef364b9dfa None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:27.668012 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e6cc3-d4ff-43bf-a4f7-6ef4ee04bb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:27.668597 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e6cc3-d4ff-43bf-a4f7-6ef4ee04bb34 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:27.668914 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e6cc3-d4ff-43bf-a4f7-6ef4ee04bb34 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:27.669198 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916e6cc3-d4ff-43bf-a4f7-6ef4ee04bb34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:27.732146 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:27.744719 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916e6cc3-d4ff-43bf-a4f7-6ef4ee04bb34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:27.746225 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916e6cc3-d4ff-43bf-a4f7-6ef4ee04bb34 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:27.868145 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 623/1255] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:44:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 17 16:44:27.874299 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3962fc0-2049-489a-9ef8-6b866ddc4e90 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:27.990158 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3962fc0-2049-489a-9ef8-6b866ddc4e90 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:28.003930 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:28.025646 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e221746-8d10-4427-8299-54df2c400ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:28.026442 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e221746-8d10-4427-8299-54df2c400ce5 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:28.026725 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e221746-8d10-4427-8299-54df2c400ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:28.027000 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e221746-8d10-4427-8299-54df2c400ce5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:28.037430 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e221746-8d10-4427-8299-54df2c400ce5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:28.038861 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e221746-8d10-4427-8299-54df2c400ce5 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:28.055933 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-7e221746-8d10-4427-8299-54df2c400ce5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:44:28.055933 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:28.055933 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:44:28.055933 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:44:28.055933 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:44:28.055933 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:44:28.055933 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:44:28.055933 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:44:28.055933 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:44:28.055933 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:44:28.055933 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:44:28.055933 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:44:28.055933 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:44:28.055933 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:44:28.055933 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:44:28.057041 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:44:28.057041 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:44:28.057041 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:44:28.057041 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:44:28.057041 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:44:28.057041 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:44:28.057041 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:44:28.057041 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:44:28.057041 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:44:28.057041 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:44:28.057041 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:44:28.057041 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:44:28.057041 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:44:28.057041 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:44:28.057041 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:44:28.057041 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:44:28.057041 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:44:28.062492 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:44:28.062492 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:44:28.062492 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:44:28.062492 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:44:28.062492 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 633/1256] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:44:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:44:28.063248 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e6cc3-d4ff-43bf-a4f7-6ef4ee04bb34 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:28.153426 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:28.166874 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916e6cc3-d4ff-43bf-a4f7-6ef4ee04bb34 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:28.201841 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f208a-a65d-454b-86f1-a3b5a6c3bbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:28.202508 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f208a-a65d-454b-86f1-a3b5a6c3bbd3 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:28.202801 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f208a-a65d-454b-86f1-a3b5a6c3bbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:28.203145 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f208a-a65d-454b-86f1-a3b5a6c3bbd3 None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:28.203871 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026f208a-a65d-454b-86f1-a3b5a6c3bbd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:28.204181 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026f208a-a65d-454b-86f1-a3b5a6c3bbd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:28.205580 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026f208a-a65d-454b-86f1-a3b5a6c3bbd3 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:28.217537 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 624/1257] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:44:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:44:28.223464 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e221746-8d10-4427-8299-54df2c400ce5 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:28.324908 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e221746-8d10-4427-8299-54df2c400ce5 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:28.359905 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5725617-41c9-4c7c-a0b0-cf3332ee3b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:28.360703 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5725617-41c9-4c7c-a0b0-cf3332ee3b6b None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:28.361053 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5725617-41c9-4c7c-a0b0-cf3332ee3b6b None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:28.361335 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5725617-41c9-4c7c-a0b0-cf3332ee3b6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:28.368448 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5725617-41c9-4c7c-a0b0-cf3332ee3b6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:28.369977 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5725617-41c9-4c7c-a0b0-cf3332ee3b6b None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:28.384227 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-a5725617-41c9-4c7c-a0b0-cf3332ee3b6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:44:28.384227 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:28.384227 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:44:28.384227 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:44:28.384227 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:44:28.384227 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:44:28.384227 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:44:28.384227 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:44:28.384227 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:44:28.384227 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:44:28.384227 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:44:28.384227 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:44:28.384227 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:44:28.384227 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:44:28.384227 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:44:28.385332 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:44:28.385332 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:44:28.385332 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:44:28.385332 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:44:28.385332 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:44:28.385332 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:44:28.385332 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:44:28.385332 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:44:28.385332 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:44:28.385332 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:44:28.385332 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:44:28.385332 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:44:28.385332 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:44:28.385332 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:44:28.385332 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:44:28.385332 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:44:28.385332 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:44:28.386380 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:44:28.386380 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:44:28.386380 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:44:28.386380 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:44:28.386380 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:44:28.386380 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:44:28.386380 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:44:28.386380 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:44:28.386380 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 634/1258] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:44:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:44:28.391495 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f208a-a65d-454b-86f1-a3b5a6c3bbd3 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:28.493491 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f208a-a65d-454b-86f1-a3b5a6c3bbd3 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:28.527040 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ecc888-b739-49d3-a032-d7564f60f68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:28.527729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ecc888-b739-49d3-a032-d7564f60f68e None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:28.527988 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ecc888-b739-49d3-a032-d7564f60f68e None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:28.528265 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ecc888-b739-49d3-a032-d7564f60f68e None admin] PATH_INFO: `/v3/projects` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:28.529085 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ecc888-b739-49d3-a032-d7564f60f68e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:28.529405 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ecc888-b739-49d3-a032-d7564f60f68e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:28.530837 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ecc888-b739-49d3-a032-d7564f60f68e None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:28.543017 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 625/1259] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:44:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:44:28.549419 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5725617-41c9-4c7c-a0b0-cf3332ee3b6b None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:28.577107 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:28.654562 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5725617-41c9-4c7c-a0b0-cf3332ee3b6b None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:28.688562 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6bd098-b9bd-4d63-a760-350d2a05dcc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:28.689241 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6bd098-b9bd-4d63-a760-350d2a05dcc8 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:28.689524 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6bd098-b9bd-4d63-a760-350d2a05dcc8 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:28.689798 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6bd098-b9bd-4d63-a760-350d2a05dcc8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:28.690648 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6bd098-b9bd-4d63-a760-350d2a05dcc8 None admin] RBAC: Adding query filter params (role.id=1025051975f64f689effa58dc4259e36, scope.project.id=927057f9a5af4833aeac9992203a223c, user.id=8e332b0ae0f64e089a01c91df2d9f28f) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:28.691468 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6bd098-b9bd-4d63-a760-350d2a05dcc8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:28.692918 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6bd098-b9bd-4d63-a760-350d2a05dcc8 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:28.698315 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 635/1260] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:44:28 2022] GET /identity/v3/role_assignments?user.id=8e332b0ae0f64e089a01c91df2d9f28f&scope.project.id=927057f9a5af4833aeac9992203a223c&role.id=1025051975f64f689effa58dc4259e36 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:44:30.698174 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c945a-0578-4042-aefa-ca0239f63cab None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:30.698762 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c945a-0578-4042-aefa-ca0239f63cab None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:30.698870 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c945a-0578-4042-aefa-ca0239f63cab None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:30.700223 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 626/1261] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:44:30.705969 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebf54d-1246-4ed2-b1dc-6fa275245f53 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:30.707139 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebf54d-1246-4ed2-b1dc-6fa275245f53 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:30.707367 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebf54d-1246-4ed2-b1dc-6fa275245f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:30.725865 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-1eebf54d-1246-4ed2-b1dc-6fa275245f53 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:44:30.727127 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-1eebf54d-1246-4ed2-b1dc-6fa275245f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:44:30.801631 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 636/1262] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:44:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 17 16:44:30.827257 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c945a-0578-4042-aefa-ca0239f63cab None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:30.839731 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:30.938880 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c945a-0578-4042-aefa-ca0239f63cab None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:30.977899 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80495404-699b-4008-9d7e-a09515466d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:30.980648 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80495404-699b-4008-9d7e-a09515466d33 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:30.980648 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80495404-699b-4008-9d7e-a09515466d33 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:30.980648 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80495404-699b-4008-9d7e-a09515466d33 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:30.991667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80495404-699b-4008-9d7e-a09515466d33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:44:31.000860 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-80495404-699b-4008-9d7e-a09515466d33 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:44:31.000860 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:31.000860 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:44:31.000860 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:44:31.000860 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:44:31.000860 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:44:31.000860 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:44:31.000860 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:44:31.000860 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:44:31.000860 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:44:31.000860 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:44:31.000860 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:44:31.000860 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:44:31.000860 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:44:31.000860 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:44:31.000860 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:44:31.002675 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:44:31.002675 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:44:31.002675 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:44:31.002675 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:44:31.002675 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:44:31.002675 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:44:31.002675 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:44:31.002675 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:44:31.002675 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:44:31.002675 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:44:31.002675 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:44:31.002675 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:44:31.002675 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:44:31.002675 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:44:31.002675 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:44:31.002675 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:44:31.004344 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:44:31.004344 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:44:31.004344 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:44:31.004344 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:44:31.004344 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:44:31.004344 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:44:31.004344 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:44:31.004344 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:44:31.004344 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:44:31.004344 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:44:31.004344 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:44:31.004344 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:44:31.004344 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 627/1263] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:44:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:44:31.007426 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebf54d-1246-4ed2-b1dc-6fa275245f53 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:31.028493 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:31.163322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebf54d-1246-4ed2-b1dc-6fa275245f53 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:31.224596 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919c2d80-de51-44c9-9699-7701a9802f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:31.225758 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c2d80-de51-44c9-9699-7701a9802f1c None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:31.226276 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c2d80-de51-44c9-9699-7701a9802f1c None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:31.226793 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c2d80-de51-44c9-9699-7701a9802f1c None admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:31.228280 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919c2d80-de51-44c9-9699-7701a9802f1c None admin] RBAC: Adding query filter params (name=admin) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:31.228831 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919c2d80-de51-44c9-9699-7701a9802f1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:31.231425 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919c2d80-de51-44c9-9699-7701a9802f1c None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:31.247634 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 637/1264] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:44:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:44:31.255006 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80495404-699b-4008-9d7e-a09515466d33 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:31.344918 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:31.385148 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80495404-699b-4008-9d7e-a09515466d33 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:31.421228 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c8931-1037-47cc-b918-0e4ee3fda742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:31.422393 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c8931-1037-47cc-b918-0e4ee3fda742 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:31.422546 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c8931-1037-47cc-b918-0e4ee3fda742 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:31.422804 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c8931-1037-47cc-b918-0e4ee3fda742 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:31.438294 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444c8931-1037-47cc-b918-0e4ee3fda742 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:31.439787 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444c8931-1037-47cc-b918-0e4ee3fda742 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:31.460476 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-444c8931-1037-47cc-b918-0e4ee3fda742 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:44:31.460476 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:31.460476 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:44:31.460476 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:44:31.460476 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:44:31.460476 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:44:31.460476 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:44:31.460476 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:44:31.460476 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:44:31.460476 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:44:31.460476 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:44:31.460476 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:44:31.460476 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:44:31.460476 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:44:31.460476 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:44:31.463967 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:44:31.463967 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:44:31.463967 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:44:31.463967 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:44:31.463967 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:44:31.463967 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:44:31.463967 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:44:31.463967 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:44:31.463967 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:44:31.463967 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:44:31.463967 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:44:31.463967 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:44:31.463967 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:44:31.463967 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:44:31.463967 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:44:31.463967 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:44:31.463967 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:44:31.465621 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:44:31.465621 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:44:31.465621 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:44:31.465621 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:44:31.465621 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 628/1265] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:44:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:44:31.467884 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919c2d80-de51-44c9-9699-7701a9802f1c None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:31.572294 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:31.592676 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919c2d80-de51-44c9-9699-7701a9802f1c None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:31.631345 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edafbe4c-dea9-403e-a876-861bdeeb4825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:31.632059 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edafbe4c-dea9-403e-a876-861bdeeb4825 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:31.632352 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edafbe4c-dea9-403e-a876-861bdeeb4825 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:31.632611 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edafbe4c-dea9-403e-a876-861bdeeb4825 None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:31.633368 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edafbe4c-dea9-403e-a876-861bdeeb4825 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:31.633681 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edafbe4c-dea9-403e-a876-861bdeeb4825 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:31.635070 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edafbe4c-dea9-403e-a876-861bdeeb4825 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:31.648260 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 638/1266] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:44:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:44:31.654557 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c8931-1037-47cc-b918-0e4ee3fda742 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:31.766769 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c8931-1037-47cc-b918-0e4ee3fda742 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:31.788709 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:31.802946 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9220c05a-4bf1-46c3-9178-075841317140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:31.803760 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220c05a-4bf1-46c3-9178-075841317140 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:31.804045 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220c05a-4bf1-46c3-9178-075841317140 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:31.804322 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220c05a-4bf1-46c3-9178-075841317140 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:31.812711 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9220c05a-4bf1-46c3-9178-075841317140 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:31.814320 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9220c05a-4bf1-46c3-9178-075841317140 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:31.833011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-9220c05a-4bf1-46c3-9178-075841317140 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 17 16:44:31.833011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:31.833011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:44:31.833011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:44:31.833011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:44:31.833011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:44:31.833011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:44:31.833011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:44:31.833011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:44:31.833011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:44:31.833011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:44:31.833011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:44:31.833011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:44:31.833011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:44:31.833011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:44:31.833011 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:44:31.834206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:44:31.834206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:44:31.834206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:44:31.834206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:44:31.834206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:44:31.834206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:44:31.834206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:44:31.834206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:44:31.834206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:44:31.834206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:44:31.834206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:44:31.834206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:44:31.834206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:44:31.834206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:44:31.834206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:44:31.834206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:44:31.834206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:44:31.834206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:44:31.835376 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:44:31.835376 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:44:31.835376 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:44:31.835376 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:44:31.835376 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:44:31.835376 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:44:31.835376 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:44:31.835376 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:44:31.835376 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 17 16:44:31.835376 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:44:31.835376 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 629/1267] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:44:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:44:31.839667 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edafbe4c-dea9-403e-a876-861bdeeb4825 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:32.001202 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edafbe4c-dea9-403e-a876-861bdeeb4825 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:32.039471 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e768f52-2a78-4ab1-8ce0-4cff8d725574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:32.040159 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e768f52-2a78-4ab1-8ce0-4cff8d725574 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:32.040441 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e768f52-2a78-4ab1-8ce0-4cff8d725574 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:32.040714 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e768f52-2a78-4ab1-8ce0-4cff8d725574 None admin] PATH_INFO: `/v3/users` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:32.041592 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e768f52-2a78-4ab1-8ce0-4cff8d725574 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:32.041901 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e768f52-2a78-4ab1-8ce0-4cff8d725574 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:32.043412 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e768f52-2a78-4ab1-8ce0-4cff8d725574 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:32.054468 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:32.063914 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 639/1268] 10.222.0.40 () {56 vars in 1143 bytes} [Sun Jul 17 16:44:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:44:32.071595 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9220c05a-4bf1-46c3-9178-075841317140 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:32.193381 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9220c05a-4bf1-46c3-9178-075841317140 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:32.233822 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c915b9-8c05-402d-bff4-f4dbc2fbb2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:32.234638 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c915b9-8c05-402d-bff4-f4dbc2fbb2c5 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:32.234924 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c915b9-8c05-402d-bff4-f4dbc2fbb2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:32.235202 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c915b9-8c05-402d-bff4-f4dbc2fbb2c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:32.249173 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c915b9-8c05-402d-bff4-f4dbc2fbb2c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:32.253990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c915b9-8c05-402d-bff4-f4dbc2fbb2c5 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:32.271406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-13c915b9-8c05-402d-bff4-f4dbc2fbb2c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:44:32.271406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:32.271406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:44:32.271406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:44:32.271406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:44:32.271406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:44:32.271406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:44:32.271406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:44:32.271406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:44:32.271406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:44:32.271406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:44:32.271406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:44:32.271406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:44:32.271406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:44:32.271406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:44:32.273178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:44:32.273178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:44:32.273178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:44:32.273178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:44:32.273178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:44:32.273178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:44:32.273178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:44:32.273178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:44:32.273178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:44:32.273178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:44:32.273178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:44:32.273178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:44:32.273178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:44:32.273178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:44:32.273178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:44:32.273178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:44:32.273178 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:44:32.274829 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:44:32.274829 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:44:32.274829 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:44:32.274829 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:44:32.274829 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 630/1269] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:44:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:44:32.278946 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e768f52-2a78-4ab1-8ce0-4cff8d725574 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:32.391851 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e768f52-2a78-4ab1-8ce0-4cff8d725574 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:32.431337 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70a75e-a5b2-47e7-87f1-2ab06965b1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:32.431945 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70a75e-a5b2-47e7-87f1-2ab06965b1c2 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:32.432235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70a75e-a5b2-47e7-87f1-2ab06965b1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:32.432516 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70a75e-a5b2-47e7-87f1-2ab06965b1c2 None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:32.433285 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c70a75e-a5b2-47e7-87f1-2ab06965b1c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:32.433601 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c70a75e-a5b2-47e7-87f1-2ab06965b1c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:32.435250 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c70a75e-a5b2-47e7-87f1-2ab06965b1c2 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:32.448491 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 640/1270] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:44:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:44:32.457834 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c915b9-8c05-402d-bff4-f4dbc2fbb2c5 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:32.498656 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:32.580806 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c915b9-8c05-402d-bff4-f4dbc2fbb2c5 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:32.634665 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18fca80-de25-474a-b33e-c1a3d35c2538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:32.635965 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18fca80-de25-474a-b33e-c1a3d35c2538 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:32.636438 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18fca80-de25-474a-b33e-c1a3d35c2538 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:32.636948 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18fca80-de25-474a-b33e-c1a3d35c2538 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:32.649986 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18fca80-de25-474a-b33e-c1a3d35c2538 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:32.651460 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18fca80-de25-474a-b33e-c1a3d35c2538 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:32.667046 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-f18fca80-de25-474a-b33e-c1a3d35c2538 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:44:32.667046 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:32.667046 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:44:32.667046 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:44:32.667046 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:44:32.667046 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:44:32.667046 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:44:32.667046 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:44:32.667046 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:44:32.667046 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:44:32.667046 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:44:32.667046 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:44:32.667046 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:44:32.667046 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:44:32.667046 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:44:32.671890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:44:32.671890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:44:32.671890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:44:32.671890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:44:32.671890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:44:32.671890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:44:32.671890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:44:32.671890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:44:32.671890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:44:32.671890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:44:32.671890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:44:32.671890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:44:32.671890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:44:32.671890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:44:32.671890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:44:32.671890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:44:32.671890 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:44:32.673976 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:44:32.673976 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:44:32.673976 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:44:32.673976 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:44:32.673976 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:44:32.673976 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:44:32.673976 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:44:32.673976 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:44:32.673976 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 631/1271] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:44:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:44:32.676360 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70a75e-a5b2-47e7-87f1-2ab06965b1c2 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:32.759949 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:32.808937 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70a75e-a5b2-47e7-87f1-2ab06965b1c2 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:32.845604 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da23d09-d161-485a-ba8b-6ea241fb768c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:32.846428 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da23d09-d161-485a-ba8b-6ea241fb768c None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:32.846711 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da23d09-d161-485a-ba8b-6ea241fb768c None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:32.847564 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da23d09-d161-485a-ba8b-6ea241fb768c None admin] PATH_INFO: `/v3/projects` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:32.848446 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da23d09-d161-485a-ba8b-6ea241fb768c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:32.848826 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da23d09-d161-485a-ba8b-6ea241fb768c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:32.850468 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da23d09-d161-485a-ba8b-6ea241fb768c None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:32.865436 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 641/1272] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:44:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:44:32.871702 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18fca80-de25-474a-b33e-c1a3d35c2538 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:32.974220 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:32.983372 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18fca80-de25-474a-b33e-c1a3d35c2538 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:33.020323 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35e298-fa5f-437f-80d7-bbdd2f333166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:33.021393 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35e298-fa5f-437f-80d7-bbdd2f333166 None admin] REQUEST_METHOD: `PUT` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:33.021829 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35e298-fa5f-437f-80d7-bbdd2f333166 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:33.022158 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35e298-fa5f-437f-80d7-bbdd2f333166 None admin] PATH_INFO: `/v3/projects/927057f9a5af4833aeac9992203a223c/users/8e332b0ae0f64e089a01c91df2d9f28f/roles/1025051975f64f689effa58dc4259e36` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:33.024194 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c35e298-fa5f-437f-80d7-bbdd2f333166 None admin] RBAC: Authorizing `identity:create_grant(project_id=927057f9a5af4833aeac9992203a223c, user_id=8e332b0ae0f64e089a01c91df2d9f28f, role_id=1025051975f64f689effa58dc4259e36)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:33.025972 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c35e298-fa5f-437f-80d7-bbdd2f333166 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:33.142293 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 632/1273] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 17 16:44:32 2022] PUT /identity/v3/projects/927057f9a5af4833aeac9992203a223c/users/8e332b0ae0f64e089a01c91df2d9f28f/roles/1025051975f64f689effa58dc4259e36 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 17 16:44:35.477526 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada0678-9e37-474e-b8c6-dedb464078df None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:35.478055 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada0678-9e37-474e-b8c6-dedb464078df None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:35.478055 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ada0678-9e37-474e-b8c6-dedb464078df None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:35.479375 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 642/1274] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:44:35.485171 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5165eca-fd11-4760-b931-90d396ec820b None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:35.493278 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5165eca-fd11-4760-b931-90d396ec820b None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:35.493278 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5165eca-fd11-4760-b931-90d396ec820b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:35.504191 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-f5165eca-fd11-4760-b931-90d396ec820b None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:44:35.505478 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-f5165eca-fd11-4760-b931-90d396ec820b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:44:35.576025 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 633/1275] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:44:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 17 16:44:35.600750 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada0678-9e37-474e-b8c6-dedb464078df None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:35.706005 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ada0678-9e37-474e-b8c6-dedb464078df None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:35.725744 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:35.743154 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b1dca-2f92-451c-8b66-dc47ca2102d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:35.743863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b1dca-2f92-451c-8b66-dc47ca2102d5 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:35.744146 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b1dca-2f92-451c-8b66-dc47ca2102d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:35.744420 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b1dca-2f92-451c-8b66-dc47ca2102d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:35.755618 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b1dca-2f92-451c-8b66-dc47ca2102d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 17 16:44:35.763367 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-292b1dca-2f92-451c-8b66-dc47ca2102d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:44:35.763367 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:35.763367 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:44:35.763367 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:44:35.763367 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:44:35.763367 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:44:35.763367 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:44:35.763367 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:44:35.763367 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:44:35.763367 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:44:35.763367 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:44:35.763367 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:44:35.763367 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 17 16:44:35.763367 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 17 16:44:35.763367 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 17 16:44:35.763367 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 17 16:44:35.764422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 17 16:44:35.764422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 17 16:44:35.764422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 17 16:44:35.764422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 17 16:44:35.764422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:44:35.764422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:44:35.764422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:44:35.764422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:44:35.764422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:44:35.764422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:44:35.764422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:44:35.764422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:44:35.764422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:44:35.764422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:44:35.764422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:44:35.764422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:44:35.765512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:44:35.765512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:44:35.765512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:44:35.765512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:44:35.765512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 17 16:44:35.765512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 17 16:44:35.765512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 17 16:44:35.765512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 17 16:44:35.765512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 17 16:44:35.765512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 17 16:44:35.765512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 17 16:44:35.765512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:44:35.765512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 643/1276] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 17 16:44:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:44:35.771494 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5165eca-fd11-4760-b931-90d396ec820b None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:35.856707 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:35.874870 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5165eca-fd11-4760-b931-90d396ec820b None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:35.909190 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec6396e-96b9-4aac-b604-02f0d8938fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:35.910171 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6396e-96b9-4aac-b604-02f0d8938fa2 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:35.910461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6396e-96b9-4aac-b604-02f0d8938fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:35.910746 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6396e-96b9-4aac-b604-02f0d8938fa2 None admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:35.911508 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec6396e-96b9-4aac-b604-02f0d8938fa2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:35.911822 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec6396e-96b9-4aac-b604-02f0d8938fa2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:35.913283 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec6396e-96b9-4aac-b604-02f0d8938fa2 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:35.921667 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 634/1277] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 17 16:44:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:44:35.928122 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b1dca-2f92-451c-8b66-dc47ca2102d5 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:36.035682 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b1dca-2f92-451c-8b66-dc47ca2102d5 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:36.071360 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21044dc8-e104-4365-ac14-e50a40e131f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:36.072169 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21044dc8-e104-4365-ac14-e50a40e131f0 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:36.072456 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21044dc8-e104-4365-ac14-e50a40e131f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:36.072741 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21044dc8-e104-4365-ac14-e50a40e131f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:36.084154 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21044dc8-e104-4365-ac14-e50a40e131f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:36.085640 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21044dc8-e104-4365-ac14-e50a40e131f0 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:36.102953 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-21044dc8-e104-4365-ac14-e50a40e131f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:44:36.102953 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:36.102953 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:44:36.102953 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:44:36.102953 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:44:36.102953 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:44:36.102953 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:44:36.102953 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:44:36.102953 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:44:36.102953 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:44:36.102953 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:44:36.102953 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:44:36.102953 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:44:36.102953 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:44:36.102953 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:44:36.104049 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:44:36.104049 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:44:36.104049 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:44:36.104049 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:44:36.104049 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:44:36.104049 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:44:36.104049 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:44:36.104049 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:44:36.104049 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:44:36.104049 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:44:36.104049 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:44:36.104049 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:44:36.104049 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:44:36.104049 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:44:36.104049 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:44:36.104049 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:44:36.104049 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:44:36.105045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:44:36.105045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:44:36.105045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:44:36.105045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:44:36.105045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 644/1278] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:44:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:44:36.110272 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec6396e-96b9-4aac-b604-02f0d8938fa2 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:36.223307 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec6396e-96b9-4aac-b604-02f0d8938fa2 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:36.258816 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e828b6-afce-406a-a00e-ad4809fb9bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:36.259488 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e828b6-afce-406a-a00e-ad4809fb9bf5 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:36.259779 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e828b6-afce-406a-a00e-ad4809fb9bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:36.260067 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e828b6-afce-406a-a00e-ad4809fb9bf5 None admin] PATH_INFO: `/v3/domains` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:36.260824 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e828b6-afce-406a-a00e-ad4809fb9bf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:36.261137 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e828b6-afce-406a-a00e-ad4809fb9bf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:36.262515 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e828b6-afce-406a-a00e-ad4809fb9bf5 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:36.274993 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 635/1279] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:44:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:44:36.281076 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21044dc8-e104-4365-ac14-e50a40e131f0 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:36.320922 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:36.392385 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21044dc8-e104-4365-ac14-e50a40e131f0 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:36.427314 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437e0c0-db71-42a8-9443-2b4251c42ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:36.428118 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437e0c0-db71-42a8-9443-2b4251c42ad7 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:36.428402 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437e0c0-db71-42a8-9443-2b4251c42ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:36.428679 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437e0c0-db71-42a8-9443-2b4251c42ad7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:36.436136 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3437e0c0-db71-42a8-9443-2b4251c42ad7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:36.437595 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3437e0c0-db71-42a8-9443-2b4251c42ad7 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:36.452045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-3437e0c0-db71-42a8-9443-2b4251c42ad7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 17 16:44:36.452045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:36.452045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:44:36.452045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:44:36.452045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:44:36.452045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:44:36.452045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:44:36.452045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:44:36.452045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:44:36.452045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:44:36.452045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:44:36.452045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:44:36.452045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 17 16:44:36.452045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 17 16:44:36.452045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 17 16:44:36.452045 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 17 16:44:36.453193 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:44:36.453193 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:44:36.453193 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 17 16:44:36.453193 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:44:36.453193 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 17 16:44:36.453193 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 17 16:44:36.453193 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:44:36.453193 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:44:36.453193 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:44:36.453193 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:44:36.453193 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:44:36.453193 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:44:36.453193 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:44:36.453193 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:44:36.453193 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:44:36.453193 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:44:36.453193 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:44:36.453193 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:44:36.454139 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:44:36.454139 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:44:36.454139 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 17 16:44:36.454139 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 17 16:44:36.454139 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 17 16:44:36.454139 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 17 16:44:36.454139 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 17 16:44:36.454139 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 17 16:44:36.454139 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 17 16:44:36.454139 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:44:36.454139 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 645/1280] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:44:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:44:36.459476 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e828b6-afce-406a-a00e-ad4809fb9bf5 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:36.463351 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:36.569608 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e828b6-afce-406a-a00e-ad4809fb9bf5 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:36.603746 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26eb26-dd57-4398-8c1a-f10153691b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:36.604346 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26eb26-dd57-4398-8c1a-f10153691b86 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:36.604635 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26eb26-dd57-4398-8c1a-f10153691b86 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:36.604948 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26eb26-dd57-4398-8c1a-f10153691b86 None admin] PATH_INFO: `/v3/users` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:36.605787 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c26eb26-dd57-4398-8c1a-f10153691b86 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:36.606105 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c26eb26-dd57-4398-8c1a-f10153691b86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:36.607548 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c26eb26-dd57-4398-8c1a-f10153691b86 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:36.625324 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 636/1281] 10.222.0.40 () {56 vars in 1143 bytes} [Sun Jul 17 16:44:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:44:36.631324 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437e0c0-db71-42a8-9443-2b4251c42ad7 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:36.720980 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:36.731076 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437e0c0-db71-42a8-9443-2b4251c42ad7 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:36.764171 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5e572-25ec-4444-ab23-0b3fb183e96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:36.764913 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5e572-25ec-4444-ab23-0b3fb183e96d None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:36.765167 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5e572-25ec-4444-ab23-0b3fb183e96d None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:36.765502 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5e572-25ec-4444-ab23-0b3fb183e96d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:36.837316 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5e572-25ec-4444-ab23-0b3fb183e96d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:36.838770 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5e572-25ec-4444-ab23-0b3fb183e96d None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:36.928725 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:36.939231 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 646/1282] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 17 16:44:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 17 16:44:36.945446 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26eb26-dd57-4398-8c1a-f10153691b86 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:37.012330 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:37.052907 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26eb26-dd57-4398-8c1a-f10153691b86 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:37.089680 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46931753-3cd2-4ddf-9fa2-e7ac8ea05428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:37.090479 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46931753-3cd2-4ddf-9fa2-e7ac8ea05428 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:37.090763 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46931753-3cd2-4ddf-9fa2-e7ac8ea05428 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:37.091041 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46931753-3cd2-4ddf-9fa2-e7ac8ea05428 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:37.102380 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46931753-3cd2-4ddf-9fa2-e7ac8ea05428 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:37.103830 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46931753-3cd2-4ddf-9fa2-e7ac8ea05428 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:37.121160 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-46931753-3cd2-4ddf-9fa2-e7ac8ea05428 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:44:37.121160 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:37.121160 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:44:37.121160 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:44:37.121160 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:44:37.121160 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:44:37.121160 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:44:37.121160 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:44:37.121160 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:44:37.121160 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:44:37.121160 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:44:37.121160 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:44:37.121160 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 17 16:44:37.121160 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 17 16:44:37.121160 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 17 16:44:37.122240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 17 16:44:37.122240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:44:37.122240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:44:37.122240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:44:37.122240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:44:37.122240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:44:37.122240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:44:37.122240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:44:37.122240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:44:37.122240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:44:37.122240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:44:37.122240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:44:37.122240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:44:37.122240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:44:37.122240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:44:37.122240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:44:37.122240 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:44:37.123404 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 17 16:44:37.123404 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 17 16:44:37.123404 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 17 16:44:37.123404 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:44:37.123404 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 637/1283] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:44:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:44:37.129153 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5e572-25ec-4444-ab23-0b3fb183e96d None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:37.234666 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5e572-25ec-4444-ab23-0b3fb183e96d None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:37.274228 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eecc22-2425-4ed1-af50-e207f8e74475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:37.274898 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eecc22-2425-4ed1-af50-e207f8e74475 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:37.275182 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eecc22-2425-4ed1-af50-e207f8e74475 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:37.276707 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eecc22-2425-4ed1-af50-e207f8e74475 None admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:37.276707 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eecc22-2425-4ed1-af50-e207f8e74475 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:37.276707 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eecc22-2425-4ed1-af50-e207f8e74475 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:37.278075 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eecc22-2425-4ed1-af50-e207f8e74475 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:37.291009 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 647/1284] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 17 16:44:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:44:37.297282 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46931753-3cd2-4ddf-9fa2-e7ac8ea05428 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:37.401319 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46931753-3cd2-4ddf-9fa2-e7ac8ea05428 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:37.415446 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:37.436218 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1699a2-2402-466e-8246-0da12de397f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:37.437066 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1699a2-2402-466e-8246-0da12de397f7 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:37.437354 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1699a2-2402-466e-8246-0da12de397f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:37.437637 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1699a2-2402-466e-8246-0da12de397f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:37.444903 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1699a2-2402-466e-8246-0da12de397f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:37.446354 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1699a2-2402-466e-8246-0da12de397f7 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:37.461308 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application [None req-ac1699a2-2402-466e-8246-0da12de397f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:44:37.461308 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:37.461308 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:44:37.461308 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:44:37.461308 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:44:37.461308 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:44:37.461308 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:44:37.461308 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:44:37.461308 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:44:37.461308 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:44:37.461308 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:44:37.461308 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:44:37.461308 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 17 16:44:37.461308 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 17 16:44:37.461308 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 17 16:44:37.462451 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 17 16:44:37.462451 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:44:37.462451 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:44:37.462451 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:44:37.462451 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:44:37.462451 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:44:37.462451 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:44:37.462451 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:44:37.462451 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application with Lock( Jul 17 16:44:37.462451 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:44:37.462451 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:44:37.462451 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:44:37.462451 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:44:37.462451 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:44:37.462451 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:44:37.462451 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:44:37.462451 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:44:37.463603 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 17 16:44:37.463603 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 17 16:44:37.463603 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 17 16:44:37.463603 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 17 16:44:37.463603 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 17 16:44:37.463603 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 17 16:44:37.463603 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 17 16:44:37.463603 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: ERROR keystone.server.flask.application Jul 17 16:44:37.463603 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 638/1285] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 17 16:44:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:44:37.468472 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eecc22-2425-4ed1-af50-e207f8e74475 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:37.520556 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:37.571401 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eecc22-2425-4ed1-af50-e207f8e74475 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:37.605312 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17926a0-44b8-41a9-b03a-2c751739676b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:37.605987 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17926a0-44b8-41a9-b03a-2c751739676b None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:37.606279 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17926a0-44b8-41a9-b03a-2c751739676b None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:37.606612 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17926a0-44b8-41a9-b03a-2c751739676b None admin] PATH_INFO: `/v3/projects` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:37.607427 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17926a0-44b8-41a9-b03a-2c751739676b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:37.607741 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17926a0-44b8-41a9-b03a-2c751739676b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:37.609228 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17926a0-44b8-41a9-b03a-2c751739676b None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:37.621822 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 648/1286] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 17 16:44:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:44:37.628658 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1699a2-2402-466e-8246-0da12de397f7 None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:37.730176 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1699a2-2402-466e-8246-0da12de397f7 None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:37.764748 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc84d05-3d00-4f07-b0a9-524f55ec18f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:37.765426 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc84d05-3d00-4f07-b0a9-524f55ec18f1 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:37.765713 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc84d05-3d00-4f07-b0a9-524f55ec18f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:37.765994 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc84d05-3d00-4f07-b0a9-524f55ec18f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:37.766824 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc84d05-3d00-4f07-b0a9-524f55ec18f1 None admin] RBAC: Adding query filter params (role.id=1025051975f64f689effa58dc4259e36, scope.project.id=927057f9a5af4833aeac9992203a223c, user.id=8e332b0ae0f64e089a01c91df2d9f28f) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:37.767140 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc84d05-3d00-4f07-b0a9-524f55ec18f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:37.768646 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc84d05-3d00-4f07-b0a9-524f55ec18f1 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:37.779212 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 639/1287] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 17 16:44:37 2022] GET /identity/v3/role_assignments?user.id=8e332b0ae0f64e089a01c91df2d9f28f&scope.project.id=927057f9a5af4833aeac9992203a223c&role.id=1025051975f64f689effa58dc4259e36 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:44:39.857238 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582ad9e-0854-4eb4-be0d-0de0a0e70fb0 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:39.857850 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582ad9e-0854-4eb4-be0d-0de0a0e70fb0 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:39.857850 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582ad9e-0854-4eb4-be0d-0de0a0e70fb0 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:39.859313 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 649/1288] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:44:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:44:39.865143 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89fb97-4fec-4621-83ee-baebd6dbbad7 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:39.865549 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89fb97-4fec-4621-83ee-baebd6dbbad7 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:39.865698 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89fb97-4fec-4621-83ee-baebd6dbbad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:39.883050 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-8e89fb97-4fec-4621-83ee-baebd6dbbad7 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:44:39.884377 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-8e89fb97-4fec-4621-83ee-baebd6dbbad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:44:39.914543 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:39.957354 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 640/1289] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:44:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:44:39.979711 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582ad9e-0854-4eb4-be0d-0de0a0e70fb0 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:40.085348 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582ad9e-0854-4eb4-be0d-0de0a0e70fb0 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:40.096026 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:40.119879 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25af3c55-3c1a-4001-b5e6-a2e7edc5814f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:40.120609 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af3c55-3c1a-4001-b5e6-a2e7edc5814f None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:40.120915 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af3c55-3c1a-4001-b5e6-a2e7edc5814f None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:40.121212 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af3c55-3c1a-4001-b5e6-a2e7edc5814f None admin] PATH_INFO: `/v3/services/placement` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:40.121851 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25af3c55-3c1a-4001-b5e6-a2e7edc5814f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:40.123214 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25af3c55-3c1a-4001-b5e6-a2e7edc5814f None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:40.135225 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-25af3c55-3c1a-4001-b5e6-a2e7edc5814f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 17 16:44:40.135225 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:40.135225 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 17 16:44:40.135225 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 17 16:44:40.135225 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 17 16:44:40.135225 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 17 16:44:40.135225 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 17 16:44:40.135225 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:44:40.135225 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 17 16:44:40.135225 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:44:40.135225 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 17 16:44:40.135225 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:44:40.135225 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:40.135225 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:44:40.135225 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:44:40.135225 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:44:40.135225 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:44:40.139759 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:44:40.139759 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:44:40.139759 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:44:40.139759 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:44:40.139759 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:44:40.139759 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:44:40.139759 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 17 16:44:40.139759 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 17 16:44:40.139759 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 17 16:44:40.139759 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 17 16:44:40.139759 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:44:40.139759 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:44:40.139759 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:44:40.139759 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:44:40.139759 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:44:40.139759 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:44:40.140889 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:44:40.140889 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:44:40.140889 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:44:40.140889 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:44:40.140889 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:44:40.140889 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:44:40.140889 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:44:40.140889 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:44:40.140889 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:44:40.140889 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:44:40.140889 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 17 16:44:40.140889 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:44:40.140889 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 17 16:44:40.140889 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:44:40.140889 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 650/1290] 10.222.0.40 () {56 vars in 1117 bytes} [Sun Jul 17 16:44:39 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:44:40.142492 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89fb97-4fec-4621-83ee-baebd6dbbad7 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:40.263731 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89fb97-4fec-4621-83ee-baebd6dbbad7 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:40.299154 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc63059-cc91-4446-a616-bfa58ec089c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:40.299874 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc63059-cc91-4446-a616-bfa58ec089c7 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:40.300891 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc63059-cc91-4446-a616-bfa58ec089c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:40.301179 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc63059-cc91-4446-a616-bfa58ec089c7 None admin] PATH_INFO: `/v3/services` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:40.301948 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc63059-cc91-4446-a616-bfa58ec089c7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:40.302312 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc63059-cc91-4446-a616-bfa58ec089c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:40.303998 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc63059-cc91-4446-a616-bfa58ec089c7 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:40.309384 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 641/1291] 10.222.0.40 () {56 vars in 1116 bytes} [Sun Jul 17 16:44:40 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:44:40.316095 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25af3c55-3c1a-4001-b5e6-a2e7edc5814f None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:40.421746 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25af3c55-3c1a-4001-b5e6-a2e7edc5814f None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:40.455870 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697605f0-30cc-44b6-92b2-eeb9c3a4fd4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:40.456640 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697605f0-30cc-44b6-92b2-eeb9c3a4fd4f None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:40.456846 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697605f0-30cc-44b6-92b2-eeb9c3a4fd4f None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:40.457130 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697605f0-30cc-44b6-92b2-eeb9c3a4fd4f None admin] PATH_INFO: `/v3/services` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:40.457866 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697605f0-30cc-44b6-92b2-eeb9c3a4fd4f None admin] RBAC: Adding query filter params (type=placement) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:40.458172 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697605f0-30cc-44b6-92b2-eeb9c3a4fd4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:40.459549 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697605f0-30cc-44b6-92b2-eeb9c3a4fd4f None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:40.463990 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 651/1292] 10.222.0.40 () {56 vars in 1116 bytes} [Sun Jul 17 16:44:40 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:44:42.519760 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad9445b-c16a-405d-a5ac-32247ffb5018 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:42.520280 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad9445b-c16a-405d-a5ac-32247ffb5018 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:42.520401 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad9445b-c16a-405d-a5ac-32247ffb5018 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:42.521720 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 642/1293] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:44:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:44:42.527074 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37169306-2e78-4345-b39c-0e6efef882f7 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:42.527494 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37169306-2e78-4345-b39c-0e6efef882f7 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:42.527667 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37169306-2e78-4345-b39c-0e6efef882f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:42.545663 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-37169306-2e78-4345-b39c-0e6efef882f7 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:44:42.546900 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-37169306-2e78-4345-b39c-0e6efef882f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:44:42.579583 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:42.619579 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 652/1294] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:44:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:44:42.642056 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad9445b-c16a-405d-a5ac-32247ffb5018 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:42.674908 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:42.749201 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad9445b-c16a-405d-a5ac-32247ffb5018 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:42.785500 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5600399-43cc-4e00-857e-98b5e2cc0472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:42.786150 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5600399-43cc-4e00-857e-98b5e2cc0472 None admin] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:42.786452 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5600399-43cc-4e00-857e-98b5e2cc0472 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:42.786712 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5600399-43cc-4e00-857e-98b5e2cc0472 None admin] PATH_INFO: `/v3/services` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:42.787447 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5600399-43cc-4e00-857e-98b5e2cc0472 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:42.788809 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5600399-43cc-4e00-857e-98b5e2cc0472 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:42.866690 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 643/1295] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Jul 17 16:44:42 2022] POST /identity/v3/services => generated 246 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 17 16:44:44.938777 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613921f0-ac1e-4e9d-aa72-4eae5bb96171 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:44.939356 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613921f0-ac1e-4e9d-aa72-4eae5bb96171 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:44.939356 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613921f0-ac1e-4e9d-aa72-4eae5bb96171 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:44.940877 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 653/1296] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:44:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:44:44.946310 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b5a12-1c03-47e6-ace0-d1b9da6f5a66 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:44.946699 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b5a12-1c03-47e6-ace0-d1b9da6f5a66 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:44.946904 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b5a12-1c03-47e6-ace0-d1b9da6f5a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:44.963852 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-878b5a12-1c03-47e6-ace0-d1b9da6f5a66 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:44:44.965091 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-878b5a12-1c03-47e6-ace0-d1b9da6f5a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:44:45.040598 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 644/1297] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:44:44 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:44:45.063716 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613921f0-ac1e-4e9d-aa72-4eae5bb96171 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:45.167024 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613921f0-ac1e-4e9d-aa72-4eae5bb96171 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:45.202201 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906fce02-afab-4cfe-aa95-03ba53ad1390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:45.202946 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906fce02-afab-4cfe-aa95-03ba53ad1390 None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:45.203229 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906fce02-afab-4cfe-aa95-03ba53ad1390 None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:45.203504 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906fce02-afab-4cfe-aa95-03ba53ad1390 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:45.204159 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906fce02-afab-4cfe-aa95-03ba53ad1390 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:45.205572 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906fce02-afab-4cfe-aa95-03ba53ad1390 None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:45.218150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-906fce02-afab-4cfe-aa95-03ba53ad1390 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 17 16:44:45.218150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:45.218150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 17 16:44:45.218150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 17 16:44:45.218150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 17 16:44:45.218150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 17 16:44:45.218150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 17 16:44:45.218150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:44:45.218150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 17 16:44:45.218150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:44:45.218150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 17 16:44:45.218150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:44:45.218150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:45.218150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:44:45.218150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:44:45.218150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:44:45.218150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:44:45.221074 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:44:45.221074 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:44:45.221074 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:44:45.221074 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:44:45.221074 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:44:45.221074 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:44:45.221074 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 17 16:44:45.221074 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 17 16:44:45.221074 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 17 16:44:45.221074 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 17 16:44:45.221074 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:44:45.221074 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:44:45.221074 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:44:45.221074 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:44:45.221074 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:44:45.221074 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:44:45.222904 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:44:45.222904 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:44:45.222904 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:44:45.222904 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:44:45.222904 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:44:45.222904 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:44:45.222904 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:44:45.222904 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:44:45.222904 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:44:45.222904 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:44:45.222904 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 17 16:44:45.222904 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:44:45.222904 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 17 16:44:45.222904 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:44:45.222904 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 654/1298] 10.222.0.40 () {56 vars in 1117 bytes} [Sun Jul 17 16:44:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:44:45.226160 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b5a12-1c03-47e6-ace0-d1b9da6f5a66 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:45.244682 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:45.326085 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b5a12-1c03-47e6-ace0-d1b9da6f5a66 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:45.362674 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b61c6-5998-4385-9837-85eb33b9a68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:45.363369 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b61c6-5998-4385-9837-85eb33b9a68a None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:45.363751 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b61c6-5998-4385-9837-85eb33b9a68a None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:45.364036 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b61c6-5998-4385-9837-85eb33b9a68a None admin] PATH_INFO: `/v3/services` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:45.364811 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1b61c6-5998-4385-9837-85eb33b9a68a None admin] RBAC: Adding query filter params (name=placement) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:45.365118 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1b61c6-5998-4385-9837-85eb33b9a68a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:45.366506 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1b61c6-5998-4385-9837-85eb33b9a68a None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:45.373681 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 645/1299] 10.222.0.40 () {56 vars in 1116 bytes} [Sun Jul 17 16:44:45 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:44:45.378720 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906fce02-afab-4cfe-aa95-03ba53ad1390 None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:45.411727 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:45.481953 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906fce02-afab-4cfe-aa95-03ba53ad1390 None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:45.521434 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af47de44-be48-4c8c-b683-c00e7ee863cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:45.522104 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47de44-be48-4c8c-b683-c00e7ee863cd None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:45.522391 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47de44-be48-4c8c-b683-c00e7ee863cd None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:45.522671 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af47de44-be48-4c8c-b683-c00e7ee863cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:45.523491 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af47de44-be48-4c8c-b683-c00e7ee863cd None admin] RBAC: Adding query filter params (interface=public, service_id=cbdda7133c1a48a0a936f079b340ef85, region_id=RegionOne) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:45.523807 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af47de44-be48-4c8c-b683-c00e7ee863cd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:45.525196 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af47de44-be48-4c8c-b683-c00e7ee863cd None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:45.530539 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 655/1300] 10.222.0.40 () {56 vars in 1251 bytes} [Sun Jul 17 16:44:45 2022] GET /identity/v3/endpoints?service_id=cbdda7133c1a48a0a936f079b340ef85&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:44:45.536951 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b61c6-5998-4385-9837-85eb33b9a68a None admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:45.630189 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:45.651226 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b61c6-5998-4385-9837-85eb33b9a68a None admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:45.690121 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a4a705-0fe7-41ec-9db2-03329cb2f387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:45.691395 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4a705-0fe7-41ec-9db2-03329cb2f387 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:45.691775 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4a705-0fe7-41ec-9db2-03329cb2f387 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:45.692098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4a705-0fe7-41ec-9db2-03329cb2f387 None admin] PATH_INFO: `/v3/services` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:45.692836 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a4a705-0fe7-41ec-9db2-03329cb2f387 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:45.694259 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a4a705-0fe7-41ec-9db2-03329cb2f387 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:45.702413 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 646/1301] 10.222.0.40 () {56 vars in 1087 bytes} [Sun Jul 17 16:44:45 2022] GET /identity/v3/services => generated 2191 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 17 16:44:47.764222 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7aa2bc-1242-4554-86b2-b8b4e7736614 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:47.764752 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7aa2bc-1242-4554-86b2-b8b4e7736614 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:47.764752 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7aa2bc-1242-4554-86b2-b8b4e7736614 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:47.766170 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 656/1302] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:44:47.772195 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba7755-94e2-4e16-828d-94629cc7be0f None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:47.773427 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba7755-94e2-4e16-828d-94629cc7be0f None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:47.773427 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba7755-94e2-4e16-828d-94629cc7be0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:47.791124 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-49ba7755-94e2-4e16-828d-94629cc7be0f None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:44:47.793155 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-49ba7755-94e2-4e16-828d-94629cc7be0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:44:47.864055 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 647/1303] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:44:47 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 17 16:44:47.887738 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7aa2bc-1242-4554-86b2-b8b4e7736614 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:47.983800 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:47.991190 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7aa2bc-1242-4554-86b2-b8b4e7736614 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:48.025514 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4d059-1a4b-443e-a8ab-89fd6658d3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:48.026325 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4d059-1a4b-443e-a8ab-89fd6658d3bb None admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:48.031560 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4d059-1a4b-443e-a8ab-89fd6658d3bb None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:48.031560 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4d059-1a4b-443e-a8ab-89fd6658d3bb None admin] PATH_INFO: `/v3/services/placement` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:48.031560 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b4d059-1a4b-443e-a8ab-89fd6658d3bb None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:48.031560 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b4d059-1a4b-443e-a8ab-89fd6658d3bb None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:48.041001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application [None req-72b4d059-1a4b-443e-a8ab-89fd6658d3bb None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 17 16:44:48.041001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:48.041001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 17 16:44:48.041001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 17 16:44:48.041001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 17 16:44:48.041001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 17 16:44:48.041001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 17 16:44:48.041001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:44:48.041001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 17 16:44:48.041001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:44:48.041001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 17 16:44:48.041001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:44:48.041001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 17 16:44:48.041001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 17 16:44:48.041001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 17 16:44:48.041001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 17 16:44:48.041001 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 17 16:44:48.042117 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 17 16:44:48.042117 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 17 16:44:48.042117 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 17 16:44:48.042117 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 17 16:44:48.042117 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 17 16:44:48.042117 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 17 16:44:48.042117 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 17 16:44:48.042117 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 17 16:44:48.042117 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 17 16:44:48.042117 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 17 16:44:48.042117 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 17 16:44:48.042117 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 17 16:44:48.042117 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 17 16:44:48.042117 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 17 16:44:48.042117 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 17 16:44:48.042117 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.get_or_create( Jul 17 16:44:48.042955 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 17 16:44:48.042955 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application with Lock( Jul 17 16:44:48.042955 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 17 16:44:48.042955 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self._enter() Jul 17 16:44:48.042955 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 17 16:44:48.042955 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 17 16:44:48.042955 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 17 16:44:48.042955 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application return self.creator() Jul 17 16:44:48.042955 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 17 16:44:48.042955 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application created_value = creator( Jul 17 16:44:48.042955 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 17 16:44:48.042955 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 17 16:44:48.042955 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 17 16:44:48.042955 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: ERROR keystone.server.flask.application Jul 17 16:44:48.042955 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 657/1304] 10.222.0.40 () {56 vars in 1117 bytes} [Sun Jul 17 16:44:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 17 16:44:48.048408 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba7755-94e2-4e16-828d-94629cc7be0f None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:48.124858 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:48.147547 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba7755-94e2-4e16-828d-94629cc7be0f None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:48.184420 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e161531-f60d-4751-84b3-93c159bd0623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:48.185392 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e161531-f60d-4751-84b3-93c159bd0623 None admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:48.185696 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e161531-f60d-4751-84b3-93c159bd0623 None admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:48.186187 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e161531-f60d-4751-84b3-93c159bd0623 None admin] PATH_INFO: `/v3/services` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:48.188312 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e161531-f60d-4751-84b3-93c159bd0623 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:44:48.188312 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e161531-f60d-4751-84b3-93c159bd0623 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:48.188501 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e161531-f60d-4751-84b3-93c159bd0623 None admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:48.194973 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 648/1305] 10.222.0.40 () {56 vars in 1116 bytes} [Sun Jul 17 16:44:48 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:44:48.200937 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4d059-1a4b-443e-a8ab-89fd6658d3bb None admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:44:48.304025 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4d059-1a4b-443e-a8ab-89fd6658d3bb None admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:44:48.339544 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfe87b-4cf0-4ddc-85d3-57eb9dfa7eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:44:48.340551 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfe87b-4cf0-4ddc-85d3-57eb9dfa7eec None admin] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:44:48.340551 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfe87b-4cf0-4ddc-85d3-57eb9dfa7eec None admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:44:48.340770 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfe87b-4cf0-4ddc-85d3-57eb9dfa7eec None admin] PATH_INFO: `/v3/endpoints` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:44:48.341543 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bfe87b-4cf0-4ddc-85d3-57eb9dfa7eec None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:44:48.342907 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bfe87b-4cf0-4ddc-85d3-57eb9dfa7eec None admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:44:48.423409 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 658/1306] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 17 16:44:48 2022] POST /identity/v3/endpoints => generated 331 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:44:58.190498 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=30 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:44:58.417445 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:52:14.423788 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0d7c5-ae14-47e0-ab49-75bb182baece None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:14.424249 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0d7c5-ae14-47e0-ab49-75bb182baece None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:14.424472 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0d7c5-ae14-47e0-ab49-75bb182baece None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:14.431612 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b6ab0-3425-4a18-a5e6-59f09b3e1e31 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:14.432020 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b6ab0-3425-4a18-a5e6-59f09b3e1e31 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:14.432308 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0b6ab0-3425-4a18-a5e6-59f09b3e1e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:14.436514 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 649/1307] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:52:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:52:14.554750 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-fc0b6ab0-3425-4a18-a5e6-59f09b3e1e31 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:52:14.556256 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-fc0b6ab0-3425-4a18-a5e6-59f09b3e1e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:14.696164 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 659/1308] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:52:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:52:14.753845 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e5fe7-5f1e-4370-945c-24f28e5fe9dd None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:14.754169 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e5fe7-5f1e-4370-945c-24f28e5fe9dd None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:14.754409 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e5fe7-5f1e-4370-945c-24f28e5fe9dd None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:14.755950 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 650/1309] 10.222.0.40 () {54 vars in 965 bytes} [Sun Jul 17 16:52:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:52:14.762129 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4c234-8304-4f2d-90d6-a6a7043914fb None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:14.763211 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4c234-8304-4f2d-90d6-a6a7043914fb None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:14.763211 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d4c234-8304-4f2d-90d6-a6a7043914fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:14.849194 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-46d4c234-8304-4f2d-90d6-a6a7043914fb None None] MFA Rules not processed for user `9663edc8502b4aecb5317b24e4a5816b`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:52:14.850480 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-46d4c234-8304-4f2d-90d6-a6a7043914fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:14.877232 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:52:14.944260 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 660/1310] 10.222.0.40 () {58 vars in 1062 bytes} [Sun Jul 17 16:52:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:52:14.954101 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77e5fe7-5f1e-4370-945c-24f28e5fe9dd None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:52:14.955864 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-e77e5fe7-5f1e-4370-945c-24f28e5fe9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:15.084099 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77e5fe7-5f1e-4370-945c-24f28e5fe9dd None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:52:15.168443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b99a8-1f13-46b8-ba82-1de0759c1e36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9663edc8502b4aecb5317b24e4a5816b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:52:15.168443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b99a8-1f13-46b8-ba82-1de0759c1e36 service neutron] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:15.168443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b99a8-1f13-46b8-ba82-1de0759c1e36 service neutron] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:15.168443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b99a8-1f13-46b8-ba82-1de0759c1e36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:15.185064 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-a88b99a8-1f13-46b8-ba82-1de0759c1e36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:15.229835 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:52:15.290839 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b99a8-1f13-46b8-ba82-1de0759c1e36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:52:15.292345 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b99a8-1f13-46b8-ba82-1de0759c1e36 service neutron] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:52:15.314508 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 651/1311] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:52:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 17 16:52:17.689878 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e4b15-62db-46c5-a87c-0500b680b632 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:17.690392 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e4b15-62db-46c5-a87c-0500b680b632 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:17.690508 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e4b15-62db-46c5-a87c-0500b680b632 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:17.691986 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 661/1312] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:52:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:52:17.697362 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa0c1e-a38d-4074-90ac-83b482f637ea None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:17.697827 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa0c1e-a38d-4074-90ac-83b482f637ea None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:17.697919 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa0c1e-a38d-4074-90ac-83b482f637ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:17.783523 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-eaaa0c1e-a38d-4074-90ac-83b482f637ea None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:52:17.784971 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-eaaa0c1e-a38d-4074-90ac-83b482f637ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:17.905494 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 652/1313] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:52:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:52:17.947974 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e4b15-62db-46c5-a87c-0500b680b632 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:52:17.949613 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-360e4b15-62db-46c5-a87c-0500b680b632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:18.074943 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e4b15-62db-46c5-a87c-0500b680b632 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:52:18.148797 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:52:18.151411 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b3b65d-b64a-41f5-baf3-43ee77856406 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9663edc8502b4aecb5317b24e4a5816b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:52:18.152074 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3b65d-b64a-41f5-baf3-43ee77856406 service neutron] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:18.152969 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3b65d-b64a-41f5-baf3-43ee77856406 service neutron] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:18.153257 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3b65d-b64a-41f5-baf3-43ee77856406 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:18.155003 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-55b3b65d-b64a-41f5-baf3-43ee77856406 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:18.271124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b3b65d-b64a-41f5-baf3-43ee77856406 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:52:18.272568 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b3b65d-b64a-41f5-baf3-43ee77856406 service neutron] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:52:18.291518 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 662/1314] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:52:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:52:20.839281 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107b8a6-8037-48b5-b65b-c0fd493f5378 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:20.840304 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107b8a6-8037-48b5-b65b-c0fd493f5378 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:20.840590 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107b8a6-8037-48b5-b65b-c0fd493f5378 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:20.842035 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 653/1315] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:52:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:52:20.847302 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35cf6c-33b4-43c2-ad96-5da5a3c4fb68 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:20.847734 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35cf6c-33b4-43c2-ad96-5da5a3c4fb68 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:20.847872 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35cf6c-33b4-43c2-ad96-5da5a3c4fb68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:20.913692 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-bb35cf6c-33b4-43c2-ad96-5da5a3c4fb68 None None] MFA Rules not processed for user `f6118de0629f4d08aa5388377f28b7ac`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:52:20.914945 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-bb35cf6c-33b4-43c2-ad96-5da5a3c4fb68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:20.969435 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:52:21.024403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 663/1316] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:52:20 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:52:21.066332 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107b8a6-8037-48b5-b65b-c0fd493f5378 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:52:21.067903 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-5107b8a6-8037-48b5-b65b-c0fd493f5378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:21.078065 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:52:21.177807 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107b8a6-8037-48b5-b65b-c0fd493f5378 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:52:21.255756 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b949c96-e4aa-48ca-bbb7-2b64bc326295 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9663edc8502b4aecb5317b24e4a5816b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:52:21.256420 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b949c96-e4aa-48ca-bbb7-2b64bc326295 service neutron] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:21.256707 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b949c96-e4aa-48ca-bbb7-2b64bc326295 service neutron] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:21.257020 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b949c96-e4aa-48ca-bbb7-2b64bc326295 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:21.258716 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-4b949c96-e4aa-48ca-bbb7-2b64bc326295 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:21.326302 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:52:21.374790 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b949c96-e4aa-48ca-bbb7-2b64bc326295 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:52:21.376317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b949c96-e4aa-48ca-bbb7-2b64bc326295 service neutron] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:52:21.396480 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 654/1317] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:52:21 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:52:24.201645 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189a89b-cceb-4615-a5c0-08d85632a0c1 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:24.202125 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189a89b-cceb-4615-a5c0-08d85632a0c1 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:24.202125 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189a89b-cceb-4615-a5c0-08d85632a0c1 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:24.203575 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 664/1318] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:52:24.208982 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59fdeda-81e8-4a50-8ade-35cab1cb6bde None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:24.209422 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59fdeda-81e8-4a50-8ade-35cab1cb6bde None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:24.209539 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59fdeda-81e8-4a50-8ade-35cab1cb6bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:24.274936 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-e59fdeda-81e8-4a50-8ade-35cab1cb6bde None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:52:24.276162 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-e59fdeda-81e8-4a50-8ade-35cab1cb6bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:24.373471 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 655/1319] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:52:24 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:52:24.413859 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0189a89b-cceb-4615-a5c0-08d85632a0c1 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:52:24.415439 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-0189a89b-cceb-4615-a5c0-08d85632a0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:24.519061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0189a89b-cceb-4615-a5c0-08d85632a0c1 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:52:24.601333 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6277a2b-77aa-4203-a283-6c6c3a754a15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9663edc8502b4aecb5317b24e4a5816b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:52:24.602025 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6277a2b-77aa-4203-a283-6c6c3a754a15 service neutron] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:24.602376 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6277a2b-77aa-4203-a283-6c6c3a754a15 service neutron] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:24.602582 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6277a2b-77aa-4203-a283-6c6c3a754a15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:24.604457 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-d6277a2b-77aa-4203-a283-6c6c3a754a15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:24.614769 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:52:24.733529 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6277a2b-77aa-4203-a283-6c6c3a754a15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:52:24.734992 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6277a2b-77aa-4203-a283-6c6c3a754a15 service neutron] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:52:24.753824 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 665/1320] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:52:24 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:52:28.064238 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc36e9-0ae8-4c4d-b654-73661864d947 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:28.064884 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc36e9-0ae8-4c4d-b654-73661864d947 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:28.064884 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc36e9-0ae8-4c4d-b654-73661864d947 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:28.066204 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 656/1321] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:52:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:52:28.071421 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075c965-d1a7-4c80-b1aa-31cb3026a9a9 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:28.071882 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075c965-d1a7-4c80-b1aa-31cb3026a9a9 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:28.072103 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075c965-d1a7-4c80-b1aa-31cb3026a9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:28.139512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-a075c965-d1a7-4c80-b1aa-31cb3026a9a9 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:52:28.140727 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-a075c965-d1a7-4c80-b1aa-31cb3026a9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:28.212574 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:52:28.260726 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 666/1322] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:52:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:52:28.301169 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc36e9-0ae8-4c4d-b654-73661864d947 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:52:28.303043 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-09fc36e9-0ae8-4c4d-b654-73661864d947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:28.325772 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:52:28.415608 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc36e9-0ae8-4c4d-b654-73661864d947 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:52:28.501936 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf8b38-2065-4d07-aa77-f35c06e36192 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9663edc8502b4aecb5317b24e4a5816b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:52:28.502513 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf8b38-2065-4d07-aa77-f35c06e36192 service neutron] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:28.502796 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf8b38-2065-4d07-aa77-f35c06e36192 service neutron] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:28.503070 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf8b38-2065-4d07-aa77-f35c06e36192 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:28.504728 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-d4cf8b38-2065-4d07-aa77-f35c06e36192 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:28.578345 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:52:28.618598 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf8b38-2065-4d07-aa77-f35c06e36192 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:52:28.619954 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf8b38-2065-4d07-aa77-f35c06e36192 service neutron] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:52:28.641039 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 657/1323] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:52:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:52:30.750281 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0423b-098f-4598-a8f6-4f13c1eefa56 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:30.750803 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0423b-098f-4598-a8f6-4f13c1eefa56 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:30.750803 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0423b-098f-4598-a8f6-4f13c1eefa56 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:30.752118 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 667/1324] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:52:30.757636 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e268a0c-dd5c-4af1-a6dd-cfcb28391342 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:30.758086 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e268a0c-dd5c-4af1-a6dd-cfcb28391342 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:30.758182 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e268a0c-dd5c-4af1-a6dd-cfcb28391342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:30.825100 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-0e268a0c-dd5c-4af1-a6dd-cfcb28391342 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:52:30.826327 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-0e268a0c-dd5c-4af1-a6dd-cfcb28391342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:30.923180 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 658/1325] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:52:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 17 16:52:30.961384 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e0423b-098f-4598-a8f6-4f13c1eefa56 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:52:30.962962 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-f6e0423b-098f-4598-a8f6-4f13c1eefa56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:31.083895 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e0423b-098f-4598-a8f6-4f13c1eefa56 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:52:31.166352 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367cbd9c-8e7d-4010-9389-220b79bf21a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9663edc8502b4aecb5317b24e4a5816b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:52:31.167013 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367cbd9c-8e7d-4010-9389-220b79bf21a6 service neutron] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:31.167863 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367cbd9c-8e7d-4010-9389-220b79bf21a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:31.168141 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367cbd9c-8e7d-4010-9389-220b79bf21a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:31.169958 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-367cbd9c-8e7d-4010-9389-220b79bf21a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:31.189770 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:52:31.288333 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367cbd9c-8e7d-4010-9389-220b79bf21a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:52:31.289858 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367cbd9c-8e7d-4010-9389-220b79bf21a6 service neutron] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:52:31.310107 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 668/1326] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:52:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:52:33.418806 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e689f-0e0f-4d66-9866-43daa93f234b None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:33.419328 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e689f-0e0f-4d66-9866-43daa93f234b None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:33.419414 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e689f-0e0f-4d66-9866-43daa93f234b None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:33.420854 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 659/1327] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:52:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:52:33.427161 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f2aa6-d28a-4608-8bfc-518ed7c7b7c9 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:33.427945 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f2aa6-d28a-4608-8bfc-518ed7c7b7c9 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:33.427945 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f2aa6-d28a-4608-8bfc-518ed7c7b7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:33.508329 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-275f2aa6-d28a-4608-8bfc-518ed7c7b7c9 None None] MFA Rules not processed for user `f6118de0629f4d08aa5388377f28b7ac`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:52:33.509621 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-275f2aa6-d28a-4608-8bfc-518ed7c7b7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:33.581475 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:52:33.618394 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 669/1328] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:52:33 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:52:33.659497 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e689f-0e0f-4d66-9866-43daa93f234b None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:52:33.661217 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-3e9e689f-0e0f-4d66-9866-43daa93f234b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:33.694347 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:52:33.767701 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e689f-0e0f-4d66-9866-43daa93f234b None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:52:33.850441 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5ac80a-4257-43db-9613-ecddb06e0430 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9663edc8502b4aecb5317b24e4a5816b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:52:33.851104 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ac80a-4257-43db-9613-ecddb06e0430 service neutron] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:33.851386 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ac80a-4257-43db-9613-ecddb06e0430 service neutron] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:33.851659 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ac80a-4257-43db-9613-ecddb06e0430 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:33.853347 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-2f5ac80a-4257-43db-9613-ecddb06e0430 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:33.937457 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:52:33.976306 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5ac80a-4257-43db-9613-ecddb06e0430 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:52:33.977786 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5ac80a-4257-43db-9613-ecddb06e0430 service neutron] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:52:34.005516 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 660/1329] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:52:33 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:52:36.632324 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b0ed1-94b8-4f36-a4a0-24329e453ce0 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:36.632931 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b0ed1-94b8-4f36-a4a0-24329e453ce0 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:36.632931 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b0ed1-94b8-4f36-a4a0-24329e453ce0 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:36.634318 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 670/1330] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:52:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:52:36.640602 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab8749-b342-4892-81ba-f713d88845d3 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:36.641120 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab8749-b342-4892-81ba-f713d88845d3 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:36.641252 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab8749-b342-4892-81ba-f713d88845d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:36.711030 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-f1ab8749-b342-4892-81ba-f713d88845d3 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:52:36.712284 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-f1ab8749-b342-4892-81ba-f713d88845d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:36.813605 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 661/1331] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:52:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:52:36.851380 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b0ed1-94b8-4f36-a4a0-24329e453ce0 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:52:36.853006 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-ef0b0ed1-94b8-4f36-a4a0-24329e453ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:36.959157 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b0ed1-94b8-4f36-a4a0-24329e453ce0 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:52:37.040682 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81a756-bb0d-4e02-b179-1c83913b8117 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9663edc8502b4aecb5317b24e4a5816b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:52:37.041362 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81a756-bb0d-4e02-b179-1c83913b8117 service neutron] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:37.041654 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81a756-bb0d-4e02-b179-1c83913b8117 service neutron] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:37.042097 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81a756-bb0d-4e02-b179-1c83913b8117 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:37.043754 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-ea81a756-bb0d-4e02-b179-1c83913b8117 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:37.071799 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:52:37.156357 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea81a756-bb0d-4e02-b179-1c83913b8117 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:52:37.157840 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea81a756-bb0d-4e02-b179-1c83913b8117 service neutron] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:52:37.178673 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 671/1332] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:52:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:52:39.278405 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22200288-9fa0-4a23-ac28-97cc748d2b26 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:39.278940 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22200288-9fa0-4a23-ac28-97cc748d2b26 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:39.278940 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22200288-9fa0-4a23-ac28-97cc748d2b26 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:39.280353 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 662/1333] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:52:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:52:39.286067 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c65ddc-cc45-4c17-b17f-aeaf0b9171c6 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:39.286613 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c65ddc-cc45-4c17-b17f-aeaf0b9171c6 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:39.286613 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c65ddc-cc45-4c17-b17f-aeaf0b9171c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:39.355672 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-71c65ddc-cc45-4c17-b17f-aeaf0b9171c6 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:52:39.356946 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-71c65ddc-cc45-4c17-b17f-aeaf0b9171c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:39.435056 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:52:39.458761 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 672/1334] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:52:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:52:39.500123 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22200288-9fa0-4a23-ac28-97cc748d2b26 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:52:39.501819 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-22200288-9fa0-4a23-ac28-97cc748d2b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:39.545672 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:52:39.616794 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22200288-9fa0-4a23-ac28-97cc748d2b26 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:52:39.692082 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c01830-a960-466d-acf6-b12ca0c9f8a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9663edc8502b4aecb5317b24e4a5816b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:52:39.692648 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c01830-a960-466d-acf6-b12ca0c9f8a8 service neutron] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:39.692965 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c01830-a960-466d-acf6-b12ca0c9f8a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:39.693251 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c01830-a960-466d-acf6-b12ca0c9f8a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:39.694930 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-54c01830-a960-466d-acf6-b12ca0c9f8a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:39.781502 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:52:39.808154 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c01830-a960-466d-acf6-b12ca0c9f8a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:52:39.809865 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c01830-a960-466d-acf6-b12ca0c9f8a8 service neutron] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:52:39.830502 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 663/1335] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:52:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:52:42.503733 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29a54f-8308-49c9-a0a2-347e95364ee0 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:42.504358 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29a54f-8308-49c9-a0a2-347e95364ee0 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:42.504503 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29a54f-8308-49c9-a0a2-347e95364ee0 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:42.506214 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 673/1336] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:52:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:52:42.512922 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ecc60-b026-411f-8f28-d03c4c86f87f None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:42.513575 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ecc60-b026-411f-8f28-d03c4c86f87f None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:42.513575 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ecc60-b026-411f-8f28-d03c4c86f87f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:42.597761 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-435ecc60-b026-411f-8f28-d03c4c86f87f None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:52:42.599820 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-435ecc60-b026-411f-8f28-d03c4c86f87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:42.726561 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 664/1337] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:52:42 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:52:42.767199 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29a54f-8308-49c9-a0a2-347e95364ee0 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:52:42.770250 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-3c29a54f-8308-49c9-a0a2-347e95364ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:42.952970 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29a54f-8308-49c9-a0a2-347e95364ee0 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:52:43.039261 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb0a3d-9c34-451e-98c8-4dc21ba423a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9663edc8502b4aecb5317b24e4a5816b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:52:43.039985 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb0a3d-9c34-451e-98c8-4dc21ba423a5 service neutron] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:43.040313 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb0a3d-9c34-451e-98c8-4dc21ba423a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:43.040632 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb0a3d-9c34-451e-98c8-4dc21ba423a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:43.043404 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-8aeb0a3d-9c34-451e-98c8-4dc21ba423a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:43.081709 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:52:43.160820 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeb0a3d-9c34-451e-98c8-4dc21ba423a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:52:43.162289 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeb0a3d-9c34-451e-98c8-4dc21ba423a5 service neutron] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:52:43.184068 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 674/1338] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:52:42 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:52:48.059686 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0447f-3b94-4f7e-845f-470332d41973 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:48.060515 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0447f-3b94-4f7e-845f-470332d41973 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:48.060515 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0447f-3b94-4f7e-845f-470332d41973 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:48.066153 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 665/1339] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:52:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:52:48.067763 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4649bd8-1e73-414a-bc8b-b1ab3d795c23 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:48.068505 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4649bd8-1e73-414a-bc8b-b1ab3d795c23 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:48.068505 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4649bd8-1e73-414a-bc8b-b1ab3d795c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:48.180155 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-d4649bd8-1e73-414a-bc8b-b1ab3d795c23 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:52:48.181480 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-d4649bd8-1e73-414a-bc8b-b1ab3d795c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:48.335618 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:52:48.361024 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 675/1340] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:52:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 17 16:52:48.402682 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0447f-3b94-4f7e-845f-470332d41973 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:52:48.405666 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-4db0447f-3b94-4f7e-845f-470332d41973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:48.500189 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:52:48.595713 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0447f-3b94-4f7e-845f-470332d41973 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:52:48.747038 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9576fd78-7046-4554-9aa8-62796e7d35be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9663edc8502b4aecb5317b24e4a5816b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:52:48.747681 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9576fd78-7046-4554-9aa8-62796e7d35be service neutron] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:48.747973 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9576fd78-7046-4554-9aa8-62796e7d35be service neutron] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:48.748262 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9576fd78-7046-4554-9aa8-62796e7d35be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:48.750208 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-9576fd78-7046-4554-9aa8-62796e7d35be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:48.897174 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:52:48.917822 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9576fd78-7046-4554-9aa8-62796e7d35be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:52:48.919466 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9576fd78-7046-4554-9aa8-62796e7d35be service neutron] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:52:48.943957 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 666/1341] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:52:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:52:52.171725 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c8a8f-3ea0-4a56-aa6c-b6f2373ce5d5 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:52.172271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c8a8f-3ea0-4a56-aa6c-b6f2373ce5d5 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:52.172271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c8a8f-3ea0-4a56-aa6c-b6f2373ce5d5 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:52.173852 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 676/1342] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:52:52.192774 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d010898-2d3d-45d0-b8c5-7865cf5019e1 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:52.192774 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d010898-2d3d-45d0-b8c5-7865cf5019e1 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:52.192774 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d010898-2d3d-45d0-b8c5-7865cf5019e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:52.254657 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-2d010898-2d3d-45d0-b8c5-7865cf5019e1 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:52:52.255836 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-2d010898-2d3d-45d0-b8c5-7865cf5019e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:52.370366 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 667/1343] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:52:52 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:52:52.409473 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c8a8f-3ea0-4a56-aa6c-b6f2373ce5d5 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:52:52.411134 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-268c8a8f-3ea0-4a56-aa6c-b6f2373ce5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:52.529380 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c8a8f-3ea0-4a56-aa6c-b6f2373ce5d5 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:52:52.608303 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5fbdf1-2a6d-4cd7-9c10-9ab194ad87df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9663edc8502b4aecb5317b24e4a5816b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:52:52.609012 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5fbdf1-2a6d-4cd7-9c10-9ab194ad87df service neutron] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:52.609304 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5fbdf1-2a6d-4cd7-9c10-9ab194ad87df service neutron] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:52.609591 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5fbdf1-2a6d-4cd7-9c10-9ab194ad87df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:52.611307 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-0c5fbdf1-2a6d-4cd7-9c10-9ab194ad87df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:52.658777 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:52:52.729300 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5fbdf1-2a6d-4cd7-9c10-9ab194ad87df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:52:52.730758 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5fbdf1-2a6d-4cd7-9c10-9ab194ad87df service neutron] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:52:52.750847 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 677/1344] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:52:52 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:52:57.134316 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed111f1f-1924-4bd3-8aa4-17c641297067 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:57.135564 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed111f1f-1924-4bd3-8aa4-17c641297067 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:57.135564 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed111f1f-1924-4bd3-8aa4-17c641297067 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:57.136317 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 668/1345] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:52:57.141949 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f320a6-8c35-4c81-a4a5-6e81867bfb46 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:57.142421 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f320a6-8c35-4c81-a4a5-6e81867bfb46 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:57.142519 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f320a6-8c35-4c81-a4a5-6e81867bfb46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:57.217870 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-67f320a6-8c35-4c81-a4a5-6e81867bfb46 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:52:57.219786 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-67f320a6-8c35-4c81-a4a5-6e81867bfb46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:57.321623 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:52:57.325294 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 678/1346] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:52:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:52:57.380214 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8fb96d-f0cb-4dc4-ad50-55b79bcf08be None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:57.380518 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8fb96d-f0cb-4dc4-ad50-55b79bcf08be None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:57.381227 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8fb96d-f0cb-4dc4-ad50-55b79bcf08be None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:57.382353 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 669/1347] 10.222.0.40 () {54 vars in 965 bytes} [Sun Jul 17 16:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:52:57.393184 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf948dc-7ed2-47ea-be73-1ee0834ee737 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:57.393502 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf948dc-7ed2-47ea-be73-1ee0834ee737 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:57.393790 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf948dc-7ed2-47ea-be73-1ee0834ee737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:57.487465 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-2cf948dc-7ed2-47ea-be73-1ee0834ee737 None None] MFA Rules not processed for user `9663edc8502b4aecb5317b24e4a5816b`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:52:57.488702 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-2cf948dc-7ed2-47ea-be73-1ee0834ee737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:57.574972 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 679/1348] 10.222.0.40 () {58 vars in 1062 bytes} [Sun Jul 17 16:52:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:52:57.584211 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8fb96d-f0cb-4dc4-ad50-55b79bcf08be None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:52:57.585817 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-3f8fb96d-f0cb-4dc4-ad50-55b79bcf08be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:57.649767 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:52:57.700523 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8fb96d-f0cb-4dc4-ad50-55b79bcf08be None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:52:57.781182 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb32d6-5008-435e-8d58-d40f38d35f5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9663edc8502b4aecb5317b24e4a5816b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:52:57.781846 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb32d6-5008-435e-8d58-d40f38d35f5a service neutron] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:52:57.782139 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb32d6-5008-435e-8d58-d40f38d35f5a service neutron] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:52:57.782417 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb32d6-5008-435e-8d58-d40f38d35f5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:52:57.784095 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-4feb32d6-5008-435e-8d58-d40f38d35f5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:52:57.898678 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:52:57.903437 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb32d6-5008-435e-8d58-d40f38d35f5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:52:57.905067 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb32d6-5008-435e-8d58-d40f38d35f5a service neutron] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:52:57.927138 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 670/1349] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:52:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:53:03.444034 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3412fe-867b-4686-be08-76d6031b8940 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:03.444570 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3412fe-867b-4686-be08-76d6031b8940 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:03.444570 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3412fe-867b-4686-be08-76d6031b8940 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:03.446295 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 680/1350] 10.222.0.40 () {54 vars in 921 bytes} [Sun Jul 17 16:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:53:03.453256 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093047b-5515-498e-9f36-8d5ce8d6ed64 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:03.454558 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093047b-5515-498e-9f36-8d5ce8d6ed64 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:03.455207 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093047b-5515-498e-9f36-8d5ce8d6ed64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:03.574513 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-4093047b-5515-498e-9f36-8d5ce8d6ed64 None None] MFA Rules not processed for user `8e332b0ae0f64e089a01c91df2d9f28f`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:53:03.575886 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-4093047b-5515-498e-9f36-8d5ce8d6ed64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:03.783403 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 671/1351] 10.222.0.40 () {58 vars in 1018 bytes} [Sun Jul 17 16:53:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:53:07.573079 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=59 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:08.293918 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251e6f9-5931-4f48-a502-a7d10a41aaa3 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:08.294364 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251e6f9-5931-4f48-a502-a7d10a41aaa3 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:08.294687 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251e6f9-5931-4f48-a502-a7d10a41aaa3 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:08.304034 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 681/1352] 10.222.0.40 () {54 vars in 921 bytes} [Sun Jul 17 16:53:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:53:08.304194 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ae25d-aae6-49af-ad4d-a924df1fe4a9 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:08.304194 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ae25d-aae6-49af-ad4d-a924df1fe4a9 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:08.304194 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53ae25d-aae6-49af-ad4d-a924df1fe4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:08.369774 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:08.395909 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-c53ae25d-aae6-49af-ad4d-a924df1fe4a9 None None] MFA Rules not processed for user `8e332b0ae0f64e089a01c91df2d9f28f`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:53:08.397461 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-c53ae25d-aae6-49af-ad4d-a924df1fe4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:08.531037 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 672/1353] 10.222.0.40 () {58 vars in 1018 bytes} [Sun Jul 17 16:53:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 17 16:53:10.217488 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93c725-1a17-47e1-977a-152b35a14de2 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:10.218054 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93c725-1a17-47e1-977a-152b35a14de2 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:10.218292 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93c725-1a17-47e1-977a-152b35a14de2 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:10.219844 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 682/1354] 10.222.0.40 () {54 vars in 921 bytes} [Sun Jul 17 16:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:53:10.230845 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe3be3d-c2d7-45bc-8fbf-5a992307ae30 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:10.230845 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe3be3d-c2d7-45bc-8fbf-5a992307ae30 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:10.230845 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe3be3d-c2d7-45bc-8fbf-5a992307ae30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:10.319442 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-dfe3be3d-c2d7-45bc-8fbf-5a992307ae30 None None] MFA Rules not processed for user `8e332b0ae0f64e089a01c91df2d9f28f`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:53:10.320728 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-dfe3be3d-c2d7-45bc-8fbf-5a992307ae30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:10.377745 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:10.456669 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 673/1355] 10.222.0.40 () {58 vars in 1018 bytes} [Sun Jul 17 16:53:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:53:13.796005 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10b4a6-b4f6-4e9f-b78a-04ee2a19cd57 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:13.796700 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10b4a6-b4f6-4e9f-b78a-04ee2a19cd57 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:13.796700 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10b4a6-b4f6-4e9f-b78a-04ee2a19cd57 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:13.799985 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 683/1356] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:53:13.805006 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc3f3d-2935-40c2-8232-0b8159a9cf09 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:13.805512 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc3f3d-2935-40c2-8232-0b8159a9cf09 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:13.805592 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc3f3d-2935-40c2-8232-0b8159a9cf09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:13.904516 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-b4bc3f3d-2935-40c2-8232-0b8159a9cf09 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:53:13.905892 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-b4bc3f3d-2935-40c2-8232-0b8159a9cf09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:14.031044 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 674/1357] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:53:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:53:14.086555 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f90485c-7a1e-4770-97f3-e0d8eceef860 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:14.087480 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f90485c-7a1e-4770-97f3-e0d8eceef860 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:14.088029 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f90485c-7a1e-4770-97f3-e0d8eceef860 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:14.090024 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 684/1358] 10.222.0.40 () {54 vars in 962 bytes} [Sun Jul 17 16:53:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:53:14.100307 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6b715-381d-4bc3-b9c7-98bc983de633 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:14.101012 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6b715-381d-4bc3-b9c7-98bc983de633 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:14.101596 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6b715-381d-4bc3-b9c7-98bc983de633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:14.119291 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:14.195061 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-4ae6b715-381d-4bc3-b9c7-98bc983de633 None None] MFA Rules not processed for user `03244087f1a34bddb11ea46aba11a148`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:53:14.197300 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-4ae6b715-381d-4bc3-b9c7-98bc983de633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:14.332019 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 675/1359] 10.222.0.40 () {58 vars in 1059 bytes} [Sun Jul 17 16:53:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 17 16:53:14.340345 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f90485c-7a1e-4770-97f3-e0d8eceef860 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:53:14.341917 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-3f90485c-7a1e-4770-97f3-e0d8eceef860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:14.499860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f90485c-7a1e-4770-97f3-e0d8eceef860 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:53:14.578270 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:14.621213 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a52fd-dc05-465e-9713-debb799cd21d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03244087f1a34bddb11ea46aba11a148', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:53:14.621599 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a52fd-dc05-465e-9713-debb799cd21d service nova] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:14.621861 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a52fd-dc05-465e-9713-debb799cd21d service nova] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:14.622150 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a52fd-dc05-465e-9713-debb799cd21d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:14.623827 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-5d2a52fd-dc05-465e-9713-debb799cd21d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:14.728454 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a52fd-dc05-465e-9713-debb799cd21d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:53:14.730117 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a52fd-dc05-465e-9713-debb799cd21d service nova] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:53:14.750789 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 685/1360] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:53:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:53:17.046470 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a7f09-24f9-4ee0-ab9a-327d82e58369 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:17.046834 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a7f09-24f9-4ee0-ab9a-327d82e58369 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:17.046926 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a7f09-24f9-4ee0-ab9a-327d82e58369 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:17.049335 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 676/1361] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:53:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:53:17.055411 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d489c9-d5a8-4443-a9e8-2ecef3e8292f None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:17.056232 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d489c9-d5a8-4443-a9e8-2ecef3e8292f None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:17.056808 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d489c9-d5a8-4443-a9e8-2ecef3e8292f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:17.147435 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-c2d489c9-d5a8-4443-a9e8-2ecef3e8292f None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:53:17.148540 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-c2d489c9-d5a8-4443-a9e8-2ecef3e8292f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:17.201796 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:17.269186 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 686/1362] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:53:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:53:17.315679 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a7f09-24f9-4ee0-ab9a-327d82e58369 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:53:17.317552 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-401a7f09-24f9-4ee0-ab9a-327d82e58369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:17.398188 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:17.491005 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a7f09-24f9-4ee0-ab9a-327d82e58369 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:53:17.643597 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fc49a0-ec4d-4e58-b2ac-12daa1a96feb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03244087f1a34bddb11ea46aba11a148', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:53:17.644304 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc49a0-ec4d-4e58-b2ac-12daa1a96feb service nova] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:17.644699 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc49a0-ec4d-4e58-b2ac-12daa1a96feb service nova] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:17.645216 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc49a0-ec4d-4e58-b2ac-12daa1a96feb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:17.646931 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-32fc49a0-ec4d-4e58-b2ac-12daa1a96feb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:17.671661 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:17.747319 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fc49a0-ec4d-4e58-b2ac-12daa1a96feb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:53:17.749242 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fc49a0-ec4d-4e58-b2ac-12daa1a96feb service nova] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:53:17.768535 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 677/1363] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:53:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 17 16:53:20.310342 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e2346-d9fe-4364-9aae-864864c6b330 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:20.310874 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e2346-d9fe-4364-9aae-864864c6b330 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:20.310978 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e2346-d9fe-4364-9aae-864864c6b330 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:20.312312 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 687/1364] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:53:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:53:20.318156 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1de31-18d6-447f-955e-d26352276e26 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:20.319051 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1de31-18d6-447f-955e-d26352276e26 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:20.319564 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1de31-18d6-447f-955e-d26352276e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:20.404107 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-d9c1de31-18d6-447f-955e-d26352276e26 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:53:20.405648 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-d9c1de31-18d6-447f-955e-d26352276e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:20.518894 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:20.528839 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 678/1365] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:53:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:53:20.572235 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0555f39-194b-4680-af74-70291deb4ffa None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:20.572471 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0555f39-194b-4680-af74-70291deb4ffa None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:20.573234 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0555f39-194b-4680-af74-70291deb4ffa None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:20.574729 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 688/1366] 10.222.0.40 () {54 vars in 962 bytes} [Sun Jul 17 16:53:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:53:20.580596 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318ec8c-dcaa-4c13-8dd2-d003f474f4cc None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:20.581111 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318ec8c-dcaa-4c13-8dd2-d003f474f4cc None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:20.581805 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318ec8c-dcaa-4c13-8dd2-d003f474f4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:20.673822 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-3318ec8c-dcaa-4c13-8dd2-d003f474f4cc None None] MFA Rules not processed for user `03244087f1a34bddb11ea46aba11a148`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:53:20.675124 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-3318ec8c-dcaa-4c13-8dd2-d003f474f4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:20.856025 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 679/1367] 10.222.0.40 () {58 vars in 1059 bytes} [Sun Jul 17 16:53:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:53:20.864536 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0555f39-194b-4680-af74-70291deb4ffa None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:53:20.866183 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-d0555f39-194b-4680-af74-70291deb4ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:21.052995 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0555f39-194b-4680-af74-70291deb4ffa None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:53:21.078193 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:21.203530 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b07960-4682-4126-967d-781c6a061cf1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03244087f1a34bddb11ea46aba11a148', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:53:21.204695 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b07960-4682-4126-967d-781c6a061cf1 service nova] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:21.205335 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b07960-4682-4126-967d-781c6a061cf1 service nova] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:21.205825 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b07960-4682-4126-967d-781c6a061cf1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:21.208324 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-32b07960-4682-4126-967d-781c6a061cf1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:21.321188 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b07960-4682-4126-967d-781c6a061cf1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:53:21.323635 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b07960-4682-4126-967d-781c6a061cf1 service nova] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:53:21.346008 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 689/1368] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:53:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:53:22.322386 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa6195-85c6-4a4d-9655-44e7ae7950d6 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:22.322933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa6195-85c6-4a4d-9655-44e7ae7950d6 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:22.322933 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa6195-85c6-4a4d-9655-44e7ae7950d6 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:22.324611 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 680/1369] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:53:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:53:22.330121 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd884d3c-493f-421d-ae65-97edc98b906c None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:22.330714 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd884d3c-493f-421d-ae65-97edc98b906c None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:22.330714 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd884d3c-493f-421d-ae65-97edc98b906c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:22.366384 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:22.409870 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-fd884d3c-493f-421d-ae65-97edc98b906c None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:53:22.411119 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-fd884d3c-493f-421d-ae65-97edc98b906c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:22.528854 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 690/1370] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:53:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:53:22.890450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175b618-ced9-47c4-8e84-7bca4aa987d7 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:22.890450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175b618-ced9-47c4-8e84-7bca4aa987d7 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:22.890450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175b618-ced9-47c4-8e84-7bca4aa987d7 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:22.890450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 681/1371] 10.222.0.40 () {54 vars in 964 bytes} [Sun Jul 17 16:53:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:53:22.897327 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09fb4d-6bd2-4e8d-98a2-df30adb27471 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:22.897327 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09fb4d-6bd2-4e8d-98a2-df30adb27471 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:22.897327 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d09fb4d-6bd2-4e8d-98a2-df30adb27471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:23.001028 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-5d09fb4d-6bd2-4e8d-98a2-df30adb27471 None None] MFA Rules not processed for user `0b7122fc744d446aa60b25844c68f15b`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:53:23.002846 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-5d09fb4d-6bd2-4e8d-98a2-df30adb27471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:23.065623 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:23.104585 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 691/1372] 10.222.0.40 () {58 vars in 1061 bytes} [Sun Jul 17 16:53:22 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:53:23.112768 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175b618-ced9-47c4-8e84-7bca4aa987d7 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:53:23.114477 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-b175b618-ced9-47c4-8e84-7bca4aa987d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:23.185737 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:23.262849 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175b618-ced9-47c4-8e84-7bca4aa987d7 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:53:23.367075 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd4932-e484-4ee6-b58a-552025c1402a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b7122fc744d446aa60b25844c68f15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:53:23.368982 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd4932-e484-4ee6-b58a-552025c1402a service cinder] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:23.368982 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd4932-e484-4ee6-b58a-552025c1402a service cinder] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:23.369241 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd4932-e484-4ee6-b58a-552025c1402a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:23.371237 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-40fd4932-e484-4ee6-b58a-552025c1402a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:23.489281 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:23.506302 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fd4932-e484-4ee6-b58a-552025c1402a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:53:23.507775 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fd4932-e484-4ee6-b58a-552025c1402a service cinder] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:53:23.533642 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 682/1373] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:53:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 17 16:53:23.671587 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85917ac8-6da0-4736-815c-cddf7ba8bfe2 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:23.672035 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85917ac8-6da0-4736-815c-cddf7ba8bfe2 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:23.672292 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85917ac8-6da0-4736-815c-cddf7ba8bfe2 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:23.674043 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 692/1374] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:53:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:53:23.679692 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8762a-e87d-409c-a741-42279096c126 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:23.679931 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8762a-e87d-409c-a741-42279096c126 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:23.680899 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8762a-e87d-409c-a741-42279096c126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:23.777948 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-67e8762a-e87d-409c-a741-42279096c126 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:53:23.779486 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-67e8762a-e87d-409c-a741-42279096c126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:23.952537 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 683/1375] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:53:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 17 16:53:23.998826 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85917ac8-6da0-4736-815c-cddf7ba8bfe2 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:53:24.000368 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-85917ac8-6da0-4736-815c-cddf7ba8bfe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:24.161999 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85917ac8-6da0-4736-815c-cddf7ba8bfe2 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:53:24.225519 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:24.297380 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eb573f-252b-4860-951a-b7ad7311b7e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03244087f1a34bddb11ea46aba11a148', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:53:24.298038 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb573f-252b-4860-951a-b7ad7311b7e8 service nova] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:24.298321 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb573f-252b-4860-951a-b7ad7311b7e8 service nova] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:24.298602 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb573f-252b-4860-951a-b7ad7311b7e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:24.300271 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-62eb573f-252b-4860-951a-b7ad7311b7e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:24.404531 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eb573f-252b-4860-951a-b7ad7311b7e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:53:24.406199 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eb573f-252b-4860-951a-b7ad7311b7e8 service nova] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:53:24.427941 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 693/1376] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:53:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:53:26.786385 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da092eb1-1d70-4504-b0af-aeaf9fa8df88 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:26.786871 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da092eb1-1d70-4504-b0af-aeaf9fa8df88 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:26.786968 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da092eb1-1d70-4504-b0af-aeaf9fa8df88 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:26.788320 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 684/1377] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:53:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:53:26.793923 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f243a2-b91a-4c74-b9f7-e4350ddfe31e None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:26.794364 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f243a2-b91a-4c74-b9f7-e4350ddfe31e None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:26.794500 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f243a2-b91a-4c74-b9f7-e4350ddfe31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:26.880176 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-72f243a2-b91a-4c74-b9f7-e4350ddfe31e None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:53:26.881459 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-72f243a2-b91a-4c74-b9f7-e4350ddfe31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:26.893393 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:27.005363 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 694/1378] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:53:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:53:27.053045 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da092eb1-1d70-4504-b0af-aeaf9fa8df88 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:53:27.054595 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-da092eb1-1d70-4504-b0af-aeaf9fa8df88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:27.124778 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:27.227761 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da092eb1-1d70-4504-b0af-aeaf9fa8df88 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:53:27.432254 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a005fd-a1ad-497a-a84a-71df9815c0d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03244087f1a34bddb11ea46aba11a148', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:53:27.432873 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a005fd-a1ad-497a-a84a-71df9815c0d7 service nova] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:27.433126 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a005fd-a1ad-497a-a84a-71df9815c0d7 service nova] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:27.433409 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a005fd-a1ad-497a-a84a-71df9815c0d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:27.435373 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-16a005fd-a1ad-497a-a84a-71df9815c0d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:27.462868 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:27.595686 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a005fd-a1ad-497a-a84a-71df9815c0d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:53:27.596389 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a005fd-a1ad-497a-a84a-71df9815c0d7 service nova] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:53:27.621466 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 685/1379] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:53:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:53:30.164080 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6741005-a634-413f-8958-ddedd7749d90 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:30.165925 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6741005-a634-413f-8958-ddedd7749d90 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:30.165925 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6741005-a634-413f-8958-ddedd7749d90 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:30.170982 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 695/1380] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:53:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:53:30.172942 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a915c00-832a-4ad1-8551-d963dd36cd17 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:30.174550 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a915c00-832a-4ad1-8551-d963dd36cd17 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:30.175214 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a915c00-832a-4ad1-8551-d963dd36cd17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:30.257532 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-9a915c00-832a-4ad1-8551-d963dd36cd17 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:53:30.259162 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-9a915c00-832a-4ad1-8551-d963dd36cd17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:30.376177 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:30.392358 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 686/1381] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:53:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:53:30.430317 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6741005-a634-413f-8958-ddedd7749d90 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:53:30.431571 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-f6741005-a634-413f-8958-ddedd7749d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:30.599629 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:30.611650 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6741005-a634-413f-8958-ddedd7749d90 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:53:30.746717 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e833a1-8de0-4f66-836b-4d2e17cb10be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03244087f1a34bddb11ea46aba11a148', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:53:30.747384 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e833a1-8de0-4f66-836b-4d2e17cb10be service nova] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:30.747673 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e833a1-8de0-4f66-836b-4d2e17cb10be service nova] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:30.747956 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e833a1-8de0-4f66-836b-4d2e17cb10be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:30.749667 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-47e833a1-8de0-4f66-836b-4d2e17cb10be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:30.860706 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e833a1-8de0-4f66-836b-4d2e17cb10be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:53:30.862367 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e833a1-8de0-4f66-836b-4d2e17cb10be service nova] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:53:30.875638 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:30.889546 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 696/1382] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:53:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 17 16:53:33.270547 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e0466-79ae-45b3-a3bb-4a8e8e5db391 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:33.271635 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e0466-79ae-45b3-a3bb-4a8e8e5db391 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:33.271635 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e0466-79ae-45b3-a3bb-4a8e8e5db391 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:33.272531 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 687/1383] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:53:33.279468 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecac262-b233-4b0a-83b9-b566d405bd3b None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:33.280329 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecac262-b233-4b0a-83b9-b566d405bd3b None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:33.280329 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecac262-b233-4b0a-83b9-b566d405bd3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:33.368325 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-aecac262-b233-4b0a-83b9-b566d405bd3b None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:53:33.371201 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-aecac262-b233-4b0a-83b9-b566d405bd3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:33.496478 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 697/1384] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:53:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:53:33.551417 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e0466-79ae-45b3-a3bb-4a8e8e5db391 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:53:33.553972 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-ea2e0466-79ae-45b3-a3bb-4a8e8e5db391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:33.756520 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e0466-79ae-45b3-a3bb-4a8e8e5db391 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:53:33.858052 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:33.933113 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff4c33-a829-490b-83ae-9caafef6b7f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03244087f1a34bddb11ea46aba11a148', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:53:33.934606 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff4c33-a829-490b-83ae-9caafef6b7f1 service nova] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:33.934606 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff4c33-a829-490b-83ae-9caafef6b7f1 service nova] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:33.934606 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff4c33-a829-490b-83ae-9caafef6b7f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:33.936493 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-82ff4c33-a829-490b-83ae-9caafef6b7f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:34.062838 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff4c33-a829-490b-83ae-9caafef6b7f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:53:34.064446 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff4c33-a829-490b-83ae-9caafef6b7f1 service nova] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:53:34.085184 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 688/1385] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:53:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 17 16:53:36.314147 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614e5f8-11a6-45b0-a3e9-081e52244ca1 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:36.314851 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614e5f8-11a6-45b0-a3e9-081e52244ca1 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:36.314851 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614e5f8-11a6-45b0-a3e9-081e52244ca1 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:36.316101 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 698/1386] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:53:36.323243 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e404e-5339-48fc-bc02-0dc53968378a None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:36.323750 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e404e-5339-48fc-bc02-0dc53968378a None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:36.324121 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e404e-5339-48fc-bc02-0dc53968378a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:36.415640 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-3d6e404e-5339-48fc-bc02-0dc53968378a None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:53:36.416946 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-3d6e404e-5339-48fc-bc02-0dc53968378a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:36.455854 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:36.537644 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 689/1387] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:53:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:53:36.587342 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614e5f8-11a6-45b0-a3e9-081e52244ca1 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:53:36.589582 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-e614e5f8-11a6-45b0-a3e9-081e52244ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:36.837868 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:36.919888 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614e5f8-11a6-45b0-a3e9-081e52244ca1 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:53:37.042582 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c8436e-9036-482e-8752-9a7d4eae6cd1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03244087f1a34bddb11ea46aba11a148', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:53:37.043247 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8436e-9036-482e-8752-9a7d4eae6cd1 service nova] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:37.043535 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8436e-9036-482e-8752-9a7d4eae6cd1 service nova] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:37.043816 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8436e-9036-482e-8752-9a7d4eae6cd1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:37.045519 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-17c8436e-9036-482e-8752-9a7d4eae6cd1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:37.089357 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:37.156749 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c8436e-9036-482e-8752-9a7d4eae6cd1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:53:37.158923 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c8436e-9036-482e-8752-9a7d4eae6cd1 service nova] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:53:37.182466 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 699/1388] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:53:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:53:39.586459 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd474e7-d489-4a94-861d-bda4dbfbc394 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:39.587115 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd474e7-d489-4a94-861d-bda4dbfbc394 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:39.587115 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd474e7-d489-4a94-861d-bda4dbfbc394 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:39.588496 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 690/1389] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:53:39.594808 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d6d89-e61f-4494-a89b-2f2a55855055 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:39.595674 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d6d89-e61f-4494-a89b-2f2a55855055 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:39.595848 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d6d89-e61f-4494-a89b-2f2a55855055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:39.629455 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6630155-d481-4c19-b35b-84e20bf87a77 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:39.629869 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6630155-d481-4c19-b35b-84e20bf87a77 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:39.630309 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6630155-d481-4c19-b35b-84e20bf87a77 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:39.632151 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 691/1390] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:53:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:53:39.638679 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b3fcd-9882-47a2-ae2e-9b4a8ae4a15a None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:39.638959 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b3fcd-9882-47a2-ae2e-9b4a8ae4a15a None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:39.639241 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b3fcd-9882-47a2-ae2e-9b4a8ae4a15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:39.672043 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-542d6d89-e61f-4494-a89b-2f2a55855055 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:53:39.673382 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-542d6d89-e61f-4494-a89b-2f2a55855055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:39.711246 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-404b3fcd-9882-47a2-ae2e-9b4a8ae4a15a None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:53:39.712508 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-404b3fcd-9882-47a2-ae2e-9b4a8ae4a15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:39.794394 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 700/1391] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:53:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:53:39.831165 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 692/1392] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:53:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 17 16:53:39.856020 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db41af5-fcba-40cb-afae-899eb990da8e None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:39.856556 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db41af5-fcba-40cb-afae-899eb990da8e None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:39.857054 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db41af5-fcba-40cb-afae-899eb990da8e None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:39.858588 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 701/1393] 10.222.0.40 () {54 vars in 961 bytes} [Sun Jul 17 16:53:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:53:39.864996 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa2672-c5b9-4313-ab5f-80384b875cec None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:39.864996 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa2672-c5b9-4313-ab5f-80384b875cec None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:39.865365 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa2672-c5b9-4313-ab5f-80384b875cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:39.879299 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db41af5-fcba-40cb-afae-899eb990da8e None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:53:39.880827 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-7db41af5-fcba-40cb-afae-899eb990da8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:39.881684 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:39.896999 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:39.952846 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-b5fa2672-c5b9-4313-ab5f-80384b875cec None None] MFA Rules not processed for user `9e13bcc0f657445cbbdea5b78563f0d3`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:53:39.954196 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-b5fa2672-c5b9-4313-ab5f-80384b875cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:40.046027 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 693/1394] 10.222.0.40 () {58 vars in 1058 bytes} [Sun Jul 17 16:53:39 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:53:40.059142 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa2672-c5b9-4313-ab5f-80384b875cec None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:53:40.061202 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db41af5-fcba-40cb-afae-899eb990da8e None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:53:40.068502 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa2672-c5b9-4313-ab5f-80384b875cec None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:53:40.156236 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd1b3c8-6f8d-4d61-94c0-2a66afa47700 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e13bcc0f657445cbbdea5b78563f0d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:53:40.156951 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1b3c8-6f8d-4d61-94c0-2a66afa47700 service glance] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:40.157243 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1b3c8-6f8d-4d61-94c0-2a66afa47700 service glance] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:40.157528 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1b3c8-6f8d-4d61-94c0-2a66afa47700 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:40.159368 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-9fd1b3c8-6f8d-4d61-94c0-2a66afa47700 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:40.180704 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:40.209593 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dfa40f-67d9-4feb-8645-254b5d405063 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03244087f1a34bddb11ea46aba11a148', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:53:40.210584 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfa40f-67d9-4feb-8645-254b5d405063 service nova] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:40.211547 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfa40f-67d9-4feb-8645-254b5d405063 service nova] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:40.212282 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfa40f-67d9-4feb-8645-254b5d405063 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:40.214858 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:40.215916 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-e2dfa40f-67d9-4feb-8645-254b5d405063 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:40.330207 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd1b3c8-6f8d-4d61-94c0-2a66afa47700 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:53:40.331843 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd1b3c8-6f8d-4d61-94c0-2a66afa47700 service glance] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:53:40.344438 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dfa40f-67d9-4feb-8645-254b5d405063 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:53:40.346099 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dfa40f-67d9-4feb-8645-254b5d405063 service nova] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:53:40.358862 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 694/1395] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:53:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 17 16:53:40.369498 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 702/1396] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:53:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 17 16:53:41.263034 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fdf7ef-4aea-4358-a49a-f540356e0716 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:41.263706 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fdf7ef-4aea-4358-a49a-f540356e0716 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:41.264012 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fdf7ef-4aea-4358-a49a-f540356e0716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:41.361555 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-16fdf7ef-4aea-4358-a49a-f540356e0716 None None] MFA Rules not processed for user `37ed5f72dc8c4a5685bd1245cadcdf7e`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:53:41.363686 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-16fdf7ef-4aea-4358-a49a-f540356e0716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:41.484981 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:41.491674 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 695/1397] 10.222.0.40 () {58 vars in 1009 bytes} [Sun Jul 17 16:53:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:53:41.552198 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28e90c-9b17-4026-b177-5ef9b3e7fdef None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:41.553071 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28e90c-9b17-4026-b177-5ef9b3e7fdef None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:41.553520 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28e90c-9b17-4026-b177-5ef9b3e7fdef None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:41.555608 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 703/1398] 10.222.0.40 () {54 vars in 943 bytes} [Sun Jul 17 16:53:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:53:41.563859 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89797b8a-db79-4b97-bcd2-5388478d09a0 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:41.564201 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89797b8a-db79-4b97-bcd2-5388478d09a0 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:41.564592 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89797b8a-db79-4b97-bcd2-5388478d09a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:41.657585 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-89797b8a-db79-4b97-bcd2-5388478d09a0 None None] MFA Rules not processed for user `69482eeb1e8547aabca9f036b5c34f13`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:53:41.659030 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-89797b8a-db79-4b97-bcd2-5388478d09a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:41.822122 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 696/1399] 10.222.0.40 () {58 vars in 1040 bytes} [Sun Jul 17 16:53:41 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 17 16:53:41.832643 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d28e90c-9b17-4026-b177-5ef9b3e7fdef None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:53:41.834330 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-8d28e90c-9b17-4026-b177-5ef9b3e7fdef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:41.963410 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:42.027888 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d28e90c-9b17-4026-b177-5ef9b3e7fdef None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:53:42.166450 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89686230-0699-48c7-a79c-31445e5b5a53 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69482eeb1e8547aabca9f036b5c34f13', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:53:42.167445 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89686230-0699-48c7-a79c-31445e5b5a53 service swift] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:42.167882 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89686230-0699-48c7-a79c-31445e5b5a53 service swift] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:42.168300 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89686230-0699-48c7-a79c-31445e5b5a53 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:42.170387 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-89686230-0699-48c7-a79c-31445e5b5a53 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:42.274673 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89686230-0699-48c7-a79c-31445e5b5a53 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:53:42.276669 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89686230-0699-48c7-a79c-31445e5b5a53 service swift] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:53:42.280995 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:42.286929 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 704/1400] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Jul 17 16:53:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 17 16:53:42.852055 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5beb40-9cbe-46c6-b6f1-d93502159820 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:42.852591 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5beb40-9cbe-46c6-b6f1-d93502159820 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:42.852591 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5beb40-9cbe-46c6-b6f1-d93502159820 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:42.858394 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 697/1401] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:53:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:53:42.859892 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344a594-ce5b-4c50-b830-d58087584259 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:42.860405 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344a594-ce5b-4c50-b830-d58087584259 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:42.860499 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344a594-ce5b-4c50-b830-d58087584259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:42.952180 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-8344a594-ce5b-4c50-b830-d58087584259 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:53:42.954433 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-8344a594-ce5b-4c50-b830-d58087584259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:43.122391 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 705/1402] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:53:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:53:43.193438 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5beb40-9cbe-46c6-b6f1-d93502159820 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:53:43.195588 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-ed5beb40-9cbe-46c6-b6f1-d93502159820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:43.288048 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:43.579253 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5beb40-9cbe-46c6-b6f1-d93502159820 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:53:43.792652 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761540d6-15b5-4e43-86fc-0afe042505aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03244087f1a34bddb11ea46aba11a148', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:53:43.794262 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761540d6-15b5-4e43-86fc-0afe042505aa service nova] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:43.795265 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761540d6-15b5-4e43-86fc-0afe042505aa service nova] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:43.795773 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761540d6-15b5-4e43-86fc-0afe042505aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:43.801924 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-761540d6-15b5-4e43-86fc-0afe042505aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:43.862045 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:43.984190 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761540d6-15b5-4e43-86fc-0afe042505aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:53:43.988156 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761540d6-15b5-4e43-86fc-0afe042505aa service nova] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:53:44.034337 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 698/1403] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:53:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 841 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:53:47.018231 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30698a97-24f1-443d-9d6b-13624262383a None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:47.019034 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30698a97-24f1-443d-9d6b-13624262383a None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:47.019239 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30698a97-24f1-443d-9d6b-13624262383a None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:47.022021 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 706/1404] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:53:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:53:47.030941 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57988e6-bf8e-4dae-a680-77c4fd535f40 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:47.031476 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57988e6-bf8e-4dae-a680-77c4fd535f40 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:47.031476 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57988e6-bf8e-4dae-a680-77c4fd535f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:47.114738 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-f57988e6-bf8e-4dae-a680-77c4fd535f40 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:53:47.116000 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-f57988e6-bf8e-4dae-a680-77c4fd535f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:47.235858 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:47.245680 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 699/1405] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:53:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:53:47.293950 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30698a97-24f1-443d-9d6b-13624262383a None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:53:47.295384 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-30698a97-24f1-443d-9d6b-13624262383a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:47.410900 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:47.500272 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30698a97-24f1-443d-9d6b-13624262383a None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:53:47.690227 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7bd38-cfaa-4bb8-9be2-24b4c79f3b94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03244087f1a34bddb11ea46aba11a148', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:53:47.690227 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7bd38-cfaa-4bb8-9be2-24b4c79f3b94 service nova] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:47.690227 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7bd38-cfaa-4bb8-9be2-24b4c79f3b94 service nova] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:47.690227 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7bd38-cfaa-4bb8-9be2-24b4c79f3b94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:47.690689 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-79d7bd38-cfaa-4bb8-9be2-24b4c79f3b94 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:47.744120 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:47.813506 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d7bd38-cfaa-4bb8-9be2-24b4c79f3b94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:53:47.815671 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d7bd38-cfaa-4bb8-9be2-24b4c79f3b94 service nova] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:53:47.839341 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 707/1406] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:53:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:53:50.208077 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f00d5-aa02-4b7e-8cb4-e93fb85d44e6 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:50.208939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f00d5-aa02-4b7e-8cb4-e93fb85d44e6 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:50.208939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f00d5-aa02-4b7e-8cb4-e93fb85d44e6 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:50.210043 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 700/1407] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:53:50.215847 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a4a94-5fb0-476e-88b9-776159f892b1 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:50.216354 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a4a94-5fb0-476e-88b9-776159f892b1 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:50.216469 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a4a94-5fb0-476e-88b9-776159f892b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:50.297467 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-9c1a4a94-5fb0-476e-88b9-776159f892b1 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:53:50.298834 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-9c1a4a94-5fb0-476e-88b9-776159f892b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:50.446294 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 708/1408] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:53:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 17 16:53:50.494486 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f00d5-aa02-4b7e-8cb4-e93fb85d44e6 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:53:50.496570 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-7d8f00d5-aa02-4b7e-8cb4-e93fb85d44e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:50.698298 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f00d5-aa02-4b7e-8cb4-e93fb85d44e6 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:53:50.814566 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:53:50.865975 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1eec73-4cd3-419c-824a-eed7d1da6be6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03244087f1a34bddb11ea46aba11a148', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:53:50.867202 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1eec73-4cd3-419c-824a-eed7d1da6be6 service nova] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:53:50.867512 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1eec73-4cd3-419c-824a-eed7d1da6be6 service nova] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:53:50.867917 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1eec73-4cd3-419c-824a-eed7d1da6be6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:53:50.869704 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-5b1eec73-4cd3-419c-824a-eed7d1da6be6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:53:50.993387 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1eec73-4cd3-419c-824a-eed7d1da6be6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:53:50.994995 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1eec73-4cd3-419c-824a-eed7d1da6be6 service nova] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:53:51.024183 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 701/1409] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:53:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:54:00.443506 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=93 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:01.020998 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=65 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:06.953037 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e951500-95db-45d5-97b7-52d58734fd5a None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:06.953643 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e951500-95db-45d5-97b7-52d58734fd5a None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:06.953849 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e951500-95db-45d5-97b7-52d58734fd5a None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:06.955581 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 709/1410] 10.222.0.40 () {54 vars in 918 bytes} [Sun Jul 17 16:54:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:54:06.963345 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b531d2c-6e1c-43d9-afee-d8005e961be4 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:06.963919 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b531d2c-6e1c-43d9-afee-d8005e961be4 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:06.963919 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b531d2c-6e1c-43d9-afee-d8005e961be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:07.056656 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-2b531d2c-6e1c-43d9-afee-d8005e961be4 None None] MFA Rules not processed for user `8e332b0ae0f64e089a01c91df2d9f28f`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:54:07.057991 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-2b531d2c-6e1c-43d9-afee-d8005e961be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:07.188095 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 702/1411] 10.222.0.40 () {58 vars in 1015 bytes} [Sun Jul 17 16:54:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:54:07.204066 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6da3ff-7830-43c1-8544-74eadbf00759 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:07.204301 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6da3ff-7830-43c1-8544-74eadbf00759 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:07.204599 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6da3ff-7830-43c1-8544-74eadbf00759 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:07.206019 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 710/1412] 10.222.0.40 () {54 vars in 918 bytes} [Sun Jul 17 16:54:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:54:07.212005 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63cde18-9ffd-4d15-b48c-682e9f0e0ffb None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:07.212301 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63cde18-9ffd-4d15-b48c-682e9f0e0ffb None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:07.212583 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63cde18-9ffd-4d15-b48c-682e9f0e0ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:07.296167 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:07.299971 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-a63cde18-9ffd-4d15-b48c-682e9f0e0ffb None None] MFA Rules not processed for user `03244087f1a34bddb11ea46aba11a148`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:54:07.301518 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-a63cde18-9ffd-4d15-b48c-682e9f0e0ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:07.436840 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 703/1413] 10.222.0.40 () {58 vars in 1015 bytes} [Sun Jul 17 16:54:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:54:10.045860 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270de18-b525-4cd9-983b-9b012de6022c None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:10.046438 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270de18-b525-4cd9-983b-9b012de6022c None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:10.046438 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270de18-b525-4cd9-983b-9b012de6022c None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:10.047786 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 711/1414] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 17 16:54:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:54:10.054090 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f646003-a5be-4dd1-b88f-c495b7ce4626 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:10.054656 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f646003-a5be-4dd1-b88f-c495b7ce4626 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:10.054656 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f646003-a5be-4dd1-b88f-c495b7ce4626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:10.138112 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-3f646003-a5be-4dd1-b88f-c495b7ce4626 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:54:10.139395 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-3f646003-a5be-4dd1-b88f-c495b7ce4626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:10.207657 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:10.255843 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 704/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 17 16:54:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:54:10.314959 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270de18-b525-4cd9-983b-9b012de6022c None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:54:10.316508 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-f270de18-b525-4cd9-983b-9b012de6022c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:10.482066 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270de18-b525-4cd9-983b-9b012de6022c None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:54:10.561447 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:10.603565 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38cd2b-3b08-4599-9a82-0e78693fdb06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03244087f1a34bddb11ea46aba11a148', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:54:10.604226 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38cd2b-3b08-4599-9a82-0e78693fdb06 service nova] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:10.604512 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38cd2b-3b08-4599-9a82-0e78693fdb06 service nova] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:10.604840 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38cd2b-3b08-4599-9a82-0e78693fdb06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:10.606518 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-1c38cd2b-3b08-4599-9a82-0e78693fdb06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:10.699971 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c38cd2b-3b08-4599-9a82-0e78693fdb06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:54:10.701619 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c38cd2b-3b08-4599-9a82-0e78693fdb06 service nova] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:54:10.720389 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 712/1416] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:54:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:54:11.664879 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c63436-a2c7-479e-a893-54948548c61c None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:11.665452 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c63436-a2c7-479e-a893-54948548c61c None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:11.665452 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c63436-a2c7-479e-a893-54948548c61c None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:11.666981 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 705/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 17 16:54:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:54:11.672479 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d02dd-5ae1-4733-bf9d-99a1c6c74e07 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:11.673127 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d02dd-5ae1-4733-bf9d-99a1c6c74e07 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:11.673233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d02dd-5ae1-4733-bf9d-99a1c6c74e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:11.764669 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-318d02dd-5ae1-4733-bf9d-99a1c6c74e07 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:54:11.765941 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-318d02dd-5ae1-4733-bf9d-99a1c6c74e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:11.796101 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:11.871198 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 713/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 17 16:54:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:54:12.028190 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c63436-a2c7-479e-a893-54948548c61c None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:54:12.029802 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-90c63436-a2c7-479e-a893-54948548c61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:12.186013 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c63436-a2c7-479e-a893-54948548c61c None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:54:12.235923 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:12.312387 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8b5dd6-1883-4744-a18e-dafaffd85832 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03244087f1a34bddb11ea46aba11a148', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:54:12.313000 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b5dd6-1883-4744-a18e-dafaffd85832 service nova] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:12.313289 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b5dd6-1883-4744-a18e-dafaffd85832 service nova] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:12.313570 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8b5dd6-1883-4744-a18e-dafaffd85832 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:12.315251 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-0a8b5dd6-1883-4744-a18e-dafaffd85832 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:12.414155 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b5dd6-1883-4744-a18e-dafaffd85832 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:54:12.415843 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8b5dd6-1883-4744-a18e-dafaffd85832 service nova] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:54:12.434990 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 706/1419] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:54:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:54:13.432325 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452b8d2-9a3f-4292-b2ba-2bdba9f01b67 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:13.432992 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452b8d2-9a3f-4292-b2ba-2bdba9f01b67 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:13.433158 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452b8d2-9a3f-4292-b2ba-2bdba9f01b67 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:13.434562 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 714/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 17 16:54:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:54:13.440561 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ba996-8977-4959-a725-ca6eb2e37234 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:13.447648 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ba996-8977-4959-a725-ca6eb2e37234 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:13.447648 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ba996-8977-4959-a725-ca6eb2e37234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:13.504248 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:13.535208 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-540ba996-8977-4959-a725-ca6eb2e37234 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:54:13.536659 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-540ba996-8977-4959-a725-ca6eb2e37234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:13.663429 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 707/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 17 16:54:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:54:13.834809 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452b8d2-9a3f-4292-b2ba-2bdba9f01b67 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:54:13.836718 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-7452b8d2-9a3f-4292-b2ba-2bdba9f01b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:14.000260 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452b8d2-9a3f-4292-b2ba-2bdba9f01b67 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:54:14.019849 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:14.129514 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f511f0b-9625-461d-ac5d-f699d9cf0b0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03244087f1a34bddb11ea46aba11a148', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:54:14.130096 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f511f0b-9625-461d-ac5d-f699d9cf0b0d service nova] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:14.130384 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f511f0b-9625-461d-ac5d-f699d9cf0b0d service nova] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:14.130662 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f511f0b-9625-461d-ac5d-f699d9cf0b0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:14.132385 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-2f511f0b-9625-461d-ac5d-f699d9cf0b0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:14.238256 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f511f0b-9625-461d-ac5d-f699d9cf0b0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:54:14.239869 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f511f0b-9625-461d-ac5d-f699d9cf0b0d service nova] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:54:14.265190 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 715/1422] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:54:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:54:15.149873 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f144091-45a1-4045-948e-ca16f0b0fe53 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:15.150417 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f144091-45a1-4045-948e-ca16f0b0fe53 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:15.150417 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f144091-45a1-4045-948e-ca16f0b0fe53 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:15.151840 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 708/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 17 16:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:54:15.157240 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688407bd-5233-44b0-8e39-f1333b2e5029 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:15.157785 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688407bd-5233-44b0-8e39-f1333b2e5029 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:15.157785 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688407bd-5233-44b0-8e39-f1333b2e5029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:15.179724 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:15.251157 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-688407bd-5233-44b0-8e39-f1333b2e5029 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:54:15.252396 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-688407bd-5233-44b0-8e39-f1333b2e5029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:15.382414 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 716/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 17 16:54:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:54:15.538872 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f144091-45a1-4045-948e-ca16f0b0fe53 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:54:15.540441 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-0f144091-45a1-4045-948e-ca16f0b0fe53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:15.667431 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:15.706451 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f144091-45a1-4045-948e-ca16f0b0fe53 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:54:15.834874 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33d6d89-8d6f-4fcd-b1a2-91dc1f0f8715 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03244087f1a34bddb11ea46aba11a148', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:54:15.835444 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d6d89-8d6f-4fcd-b1a2-91dc1f0f8715 service nova] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:15.835739 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d6d89-8d6f-4fcd-b1a2-91dc1f0f8715 service nova] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:15.836024 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33d6d89-8d6f-4fcd-b1a2-91dc1f0f8715 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:15.837743 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-a33d6d89-8d6f-4fcd-b1a2-91dc1f0f8715 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:15.925422 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:15.940614 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33d6d89-8d6f-4fcd-b1a2-91dc1f0f8715 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:54:15.942546 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33d6d89-8d6f-4fcd-b1a2-91dc1f0f8715 service nova] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:54:15.963207 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 709/1425] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:54:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:54:16.860897 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e18600-0059-4819-93fd-fbd15236aa06 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:16.861441 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e18600-0059-4819-93fd-fbd15236aa06 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:16.861441 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e18600-0059-4819-93fd-fbd15236aa06 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:16.862835 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 717/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 17 16:54:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:54:16.868347 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec4052-cbb0-481d-8706-7fd50b458727 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:16.868838 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec4052-cbb0-481d-8706-7fd50b458727 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:16.868961 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec4052-cbb0-481d-8706-7fd50b458727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:16.960509 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-b1ec4052-cbb0-481d-8706-7fd50b458727 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:54:16.961996 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-b1ec4052-cbb0-481d-8706-7fd50b458727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:17.072638 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 710/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 17 16:54:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 17 16:54:17.226667 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e18600-0059-4819-93fd-fbd15236aa06 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:54:17.228220 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-34e18600-0059-4819-93fd-fbd15236aa06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:17.325805 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:17.390307 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e18600-0059-4819-93fd-fbd15236aa06 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:54:17.519111 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b50dea-e67a-40e9-a9f2-916f42add216 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03244087f1a34bddb11ea46aba11a148', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:54:17.519776 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b50dea-e67a-40e9-a9f2-916f42add216 service nova] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:17.520061 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b50dea-e67a-40e9-a9f2-916f42add216 service nova] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:17.520339 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b50dea-e67a-40e9-a9f2-916f42add216 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:17.522770 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-31b50dea-e67a-40e9-a9f2-916f42add216 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:17.576459 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:17.621782 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b50dea-e67a-40e9-a9f2-916f42add216 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:54:17.623400 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b50dea-e67a-40e9-a9f2-916f42add216 service nova] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:54:17.643158 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 718/1428] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:54:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:54:18.544670 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb855045-0ccb-476d-8cab-ebf94d9c95ed None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:18.545251 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb855045-0ccb-476d-8cab-ebf94d9c95ed None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:18.545251 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb855045-0ccb-476d-8cab-ebf94d9c95ed None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:18.546796 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 711/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 17 16:54:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:54:18.553127 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76050a0-699a-4907-a00b-d12107d40ca6 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:18.553859 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76050a0-699a-4907-a00b-d12107d40ca6 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:18.554015 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76050a0-699a-4907-a00b-d12107d40ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:18.634364 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-a76050a0-699a-4907-a00b-d12107d40ca6 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:54:18.635597 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-a76050a0-699a-4907-a00b-d12107d40ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:18.737403 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 719/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 17 16:54:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:54:18.893465 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb855045-0ccb-476d-8cab-ebf94d9c95ed None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:54:18.895035 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-eb855045-0ccb-476d-8cab-ebf94d9c95ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:18.961828 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:19.075448 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb855045-0ccb-476d-8cab-ebf94d9c95ed None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:54:19.229660 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed1aee-558d-4bb1-8ba2-e8c8823407db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03244087f1a34bddb11ea46aba11a148', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:54:19.230412 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed1aee-558d-4bb1-8ba2-e8c8823407db service nova] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:19.230737 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed1aee-558d-4bb1-8ba2-e8c8823407db service nova] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:19.231129 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed1aee-558d-4bb1-8ba2-e8c8823407db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:19.232903 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-4aed1aee-558d-4bb1-8ba2-e8c8823407db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:19.241450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:19.347684 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed1aee-558d-4bb1-8ba2-e8c8823407db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:54:19.349987 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aed1aee-558d-4bb1-8ba2-e8c8823407db service nova] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:54:19.370475 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 712/1431] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:54:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:54:20.291637 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458ae8c-2534-41fb-90fb-bd6f1276445e None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:20.292167 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458ae8c-2534-41fb-90fb-bd6f1276445e None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:20.292167 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6458ae8c-2534-41fb-90fb-bd6f1276445e None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:20.293541 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 720/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 17 16:54:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:54:20.299599 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a187e-2ead-401f-97db-c928f88689aa None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:20.300121 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a187e-2ead-401f-97db-c928f88689aa None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:20.300232 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a187e-2ead-401f-97db-c928f88689aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:20.382007 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-544a187e-2ead-401f-97db-c928f88689aa None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:54:20.383264 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-544a187e-2ead-401f-97db-c928f88689aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:20.460971 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:20.496618 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 713/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 17 16:54:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:54:20.652491 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6458ae8c-2534-41fb-90fb-bd6f1276445e None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:54:20.653960 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-6458ae8c-2534-41fb-90fb-bd6f1276445e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:20.673024 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:20.841766 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6458ae8c-2534-41fb-90fb-bd6f1276445e None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:54:20.950835 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:20.983465 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b89c91-d1eb-4364-bb04-0211becf4483 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03244087f1a34bddb11ea46aba11a148', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:54:20.984230 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b89c91-d1eb-4364-bb04-0211becf4483 service nova] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:20.984560 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b89c91-d1eb-4364-bb04-0211becf4483 service nova] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:20.984999 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b89c91-d1eb-4364-bb04-0211becf4483 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:20.987102 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-e3b89c91-d1eb-4364-bb04-0211becf4483 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:21.096264 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b89c91-d1eb-4364-bb04-0211becf4483 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:54:21.098002 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b89c91-d1eb-4364-bb04-0211becf4483 service nova] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:54:21.119683 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 721/1434] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:54:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 17 16:54:22.107947 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e307b7d-a268-4693-a08f-68159b0583fc None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:22.108625 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e307b7d-a268-4693-a08f-68159b0583fc None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:22.108625 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e307b7d-a268-4693-a08f-68159b0583fc None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:22.110208 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 714/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 17 16:54:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:54:22.117574 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5824159-a81b-43e5-a090-7daae97389a0 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:22.118422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5824159-a81b-43e5-a090-7daae97389a0 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:22.118599 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5824159-a81b-43e5-a090-7daae97389a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:22.210170 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-d5824159-a81b-43e5-a090-7daae97389a0 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:54:22.211460 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-d5824159-a81b-43e5-a090-7daae97389a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:22.262840 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:22.335461 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 722/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 17 16:54:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 17 16:54:22.490729 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e307b7d-a268-4693-a08f-68159b0583fc None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:54:22.492195 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-9e307b7d-a268-4693-a08f-68159b0583fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:22.653188 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e307b7d-a268-4693-a08f-68159b0583fc None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:54:22.713987 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:22.778946 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0289973-a10d-46af-bf5b-98c32ac101f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03244087f1a34bddb11ea46aba11a148', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:54:22.779608 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0289973-a10d-46af-bf5b-98c32ac101f4 service nova] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:22.779889 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0289973-a10d-46af-bf5b-98c32ac101f4 service nova] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:22.780193 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0289973-a10d-46af-bf5b-98c32ac101f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:22.781882 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-e0289973-a10d-46af-bf5b-98c32ac101f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:22.877813 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0289973-a10d-46af-bf5b-98c32ac101f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:54:22.879443 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0289973-a10d-46af-bf5b-98c32ac101f4 service nova] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:54:22.899810 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 715/1437] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:54:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:54:23.822031 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3e3b7-bccf-4333-9496-dfe90d1c8854 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:23.822917 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3e3b7-bccf-4333-9496-dfe90d1c8854 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:23.823112 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3e3b7-bccf-4333-9496-dfe90d1c8854 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:23.824503 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 723/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 17 16:54:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:54:23.829801 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847bf02-ce4e-4109-a155-5d50c1a63cdf None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:23.830235 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847bf02-ce4e-4109-a155-5d50c1a63cdf None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:23.830357 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847bf02-ce4e-4109-a155-5d50c1a63cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:23.916402 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-c847bf02-ce4e-4109-a155-5d50c1a63cdf None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:54:23.918639 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-c847bf02-ce4e-4109-a155-5d50c1a63cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:23.922474 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:24.030568 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 716/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 17 16:54:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:54:24.191375 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d3e3b7-bccf-4333-9496-dfe90d1c8854 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:54:24.193848 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-40d3e3b7-bccf-4333-9496-dfe90d1c8854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:24.364269 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d3e3b7-bccf-4333-9496-dfe90d1c8854 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:54:24.393726 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:24.515879 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2ce19a-386e-4d79-8461-51505d0ebfab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03244087f1a34bddb11ea46aba11a148', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:54:24.516636 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ce19a-386e-4d79-8461-51505d0ebfab service nova] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:24.517115 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ce19a-386e-4d79-8461-51505d0ebfab service nova] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:24.517481 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ce19a-386e-4d79-8461-51505d0ebfab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:24.519986 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-ff2ce19a-386e-4d79-8461-51505d0ebfab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:24.665641 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2ce19a-386e-4d79-8461-51505d0ebfab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:54:24.667273 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2ce19a-386e-4d79-8461-51505d0ebfab service nova] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:54:24.689164 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 724/1440] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:54:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:54:26.702875 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b3722-710f-4bec-9135-d782ad47e549 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:26.703493 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b3722-710f-4bec-9135-d782ad47e549 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:26.703493 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b3722-710f-4bec-9135-d782ad47e549 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:26.704855 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 717/1441] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 17 16:54:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:54:26.711983 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310a6a4-8034-4c46-aec6-3dd139888300 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:26.712504 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310a6a4-8034-4c46-aec6-3dd139888300 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:26.712712 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7310a6a4-8034-4c46-aec6-3dd139888300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:26.767096 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:26.793836 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-7310a6a4-8034-4c46-aec6-3dd139888300 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:54:26.795081 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-7310a6a4-8034-4c46-aec6-3dd139888300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:26.911414 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 725/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 17 16:54:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:54:26.954568 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b3722-710f-4bec-9135-d782ad47e549 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:54:26.956039 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-643b3722-710f-4bec-9135-d782ad47e549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:27.080013 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b3722-710f-4bec-9135-d782ad47e549 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:54:27.117137 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:27.159857 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f64ffa-fd2d-4cd4-8a8f-3aaf53396836 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9663edc8502b4aecb5317b24e4a5816b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:54:27.160520 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f64ffa-fd2d-4cd4-8a8f-3aaf53396836 service neutron] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:27.160841 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f64ffa-fd2d-4cd4-8a8f-3aaf53396836 service neutron] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:27.161142 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f64ffa-fd2d-4cd4-8a8f-3aaf53396836 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:27.163382 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-90f64ffa-fd2d-4cd4-8a8f-3aaf53396836 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:27.293159 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f64ffa-fd2d-4cd4-8a8f-3aaf53396836 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:54:27.294610 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f64ffa-fd2d-4cd4-8a8f-3aaf53396836 service neutron] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:54:27.314872 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 718/1443] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:54:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:54:29.527632 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e57b93-342d-4e75-9ab1-444cc056df51 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:29.528651 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e57b93-342d-4e75-9ab1-444cc056df51 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:29.528651 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e57b93-342d-4e75-9ab1-444cc056df51 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:29.529689 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 726/1444] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 17 16:54:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:54:29.535111 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e4bd7-c78c-4dc6-870d-64e0c30876b5 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:29.537292 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e4bd7-c78c-4dc6-870d-64e0c30876b5 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:29.537292 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e4bd7-c78c-4dc6-870d-64e0c30876b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:29.606340 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-267e4bd7-c78c-4dc6-870d-64e0c30876b5 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:54:29.607608 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-267e4bd7-c78c-4dc6-870d-64e0c30876b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:29.621347 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:29.712209 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 719/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 17 16:54:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:54:29.754736 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e57b93-342d-4e75-9ab1-444cc056df51 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:54:29.756910 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-02e57b93-342d-4e75-9ab1-444cc056df51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:29.866289 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e57b93-342d-4e75-9ab1-444cc056df51 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:54:29.878746 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:29.947323 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ae502-8ad9-4414-836f-f7eef468c659 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9663edc8502b4aecb5317b24e4a5816b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:54:29.947978 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ae502-8ad9-4414-836f-f7eef468c659 service neutron] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:29.948261 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ae502-8ad9-4414-836f-f7eef468c659 service neutron] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:29.948534 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ae502-8ad9-4414-836f-f7eef468c659 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:29.950218 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-c33ae502-8ad9-4414-836f-f7eef468c659 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:30.070976 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33ae502-8ad9-4414-836f-f7eef468c659 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:54:30.072535 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33ae502-8ad9-4414-836f-f7eef468c659 service neutron] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:54:30.094007 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 727/1446] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:54:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:54:32.542648 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2a7ab-0b45-4b5f-b33a-4d1fb36098c4 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:32.543288 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2a7ab-0b45-4b5f-b33a-4d1fb36098c4 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:32.543288 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2a7ab-0b45-4b5f-b33a-4d1fb36098c4 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:32.544586 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 720/1447] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 17 16:54:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:54:32.550320 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ce845-838d-4bc1-a648-e8f864027d44 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:32.550784 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ce845-838d-4bc1-a648-e8f864027d44 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:32.550887 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ce845-838d-4bc1-a648-e8f864027d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:32.605770 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:32.625159 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-c84ce845-838d-4bc1-a648-e8f864027d44 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:54:32.626395 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-c84ce845-838d-4bc1-a648-e8f864027d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:32.729406 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 728/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 17 16:54:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:54:32.769686 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c2a7ab-0b45-4b5f-b33a-4d1fb36098c4 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:54:32.771558 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-06c2a7ab-0b45-4b5f-b33a-4d1fb36098c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:32.889539 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c2a7ab-0b45-4b5f-b33a-4d1fb36098c4 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:54:32.931744 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:32.965847 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee7527-5156-49dd-a3cb-d55db1ae4fcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9663edc8502b4aecb5317b24e4a5816b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:54:32.966509 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee7527-5156-49dd-a3cb-d55db1ae4fcc service neutron] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:32.966791 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee7527-5156-49dd-a3cb-d55db1ae4fcc service neutron] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:32.967070 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee7527-5156-49dd-a3cb-d55db1ae4fcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:32.968737 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-e8ee7527-5156-49dd-a3cb-d55db1ae4fcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:33.086250 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee7527-5156-49dd-a3cb-d55db1ae4fcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:54:33.087700 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee7527-5156-49dd-a3cb-d55db1ae4fcc service neutron] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:54:33.107350 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 721/1449] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:54:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:54:35.300057 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042b042-f6ce-4ac5-9cda-b5c63939ac88 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:35.300628 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042b042-f6ce-4ac5-9cda-b5c63939ac88 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:35.300628 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042b042-f6ce-4ac5-9cda-b5c63939ac88 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:35.302450 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 729/1450] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 17 16:54:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 17 16:54:35.307753 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f33653-8f33-4127-859d-374f6f76090d None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:35.308286 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f33653-8f33-4127-859d-374f6f76090d None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:35.308390 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f33653-8f33-4127-859d-374f6f76090d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:35.385710 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-97f33653-8f33-4127-859d-374f6f76090d None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:54:35.386934 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-97f33653-8f33-4127-859d-374f6f76090d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:35.408520 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:35.489896 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 722/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 17 16:54:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 17 16:54:35.530182 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0042b042-f6ce-4ac5-9cda-b5c63939ac88 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:54:35.531727 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-0042b042-f6ce-4ac5-9cda-b5c63939ac88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:35.641671 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0042b042-f6ce-4ac5-9cda-b5c63939ac88 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:54:35.662609 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:35.719628 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac87593-7bc3-4299-9ae7-4570f11cd935 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9663edc8502b4aecb5317b24e4a5816b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:54:35.720282 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac87593-7bc3-4299-9ae7-4570f11cd935 service neutron] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:35.720565 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac87593-7bc3-4299-9ae7-4570f11cd935 service neutron] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:35.720937 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac87593-7bc3-4299-9ae7-4570f11cd935 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:35.722593 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-1ac87593-7bc3-4299-9ae7-4570f11cd935 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:35.843791 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac87593-7bc3-4299-9ae7-4570f11cd935 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:54:35.845282 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac87593-7bc3-4299-9ae7-4570f11cd935 service neutron] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:54:35.864926 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 730/1452] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:54:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:54:38.087618 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604e7ab-baeb-41b0-97af-3d43874c2409 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:38.088190 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604e7ab-baeb-41b0-97af-3d43874c2409 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:38.088190 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604e7ab-baeb-41b0-97af-3d43874c2409 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:38.089594 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 723/1453] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 17 16:54:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:54:38.095203 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffce99-9d69-4333-8329-2042633b9c90 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:38.095739 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffce99-9d69-4333-8329-2042633b9c90 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:38.095739 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffce99-9d69-4333-8329-2042633b9c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:38.165917 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:38.169543 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-baffce99-9d69-4333-8329-2042633b9c90 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:54:38.171494 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-baffce99-9d69-4333-8329-2042633b9c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:38.280663 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 731/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 17 16:54:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:54:38.323219 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604e7ab-baeb-41b0-97af-3d43874c2409 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:54:38.324888 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-4604e7ab-baeb-41b0-97af-3d43874c2409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:38.436316 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604e7ab-baeb-41b0-97af-3d43874c2409 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:54:38.489440 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:38.515202 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49779df9-6487-44bf-8d58-0817b3ab0893 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9663edc8502b4aecb5317b24e4a5816b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:54:38.515882 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49779df9-6487-44bf-8d58-0817b3ab0893 service neutron] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:38.516161 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49779df9-6487-44bf-8d58-0817b3ab0893 service neutron] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:38.516445 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49779df9-6487-44bf-8d58-0817b3ab0893 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:38.518207 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-49779df9-6487-44bf-8d58-0817b3ab0893 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:38.640088 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49779df9-6487-44bf-8d58-0817b3ab0893 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:54:38.641495 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49779df9-6487-44bf-8d58-0817b3ab0893 service neutron] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:54:38.661513 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 724/1455] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:54:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:54:40.789605 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa796e5-0d94-4212-95ef-2b4d82b679c1 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:40.790173 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa796e5-0d94-4212-95ef-2b4d82b679c1 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:40.790173 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa796e5-0d94-4212-95ef-2b4d82b679c1 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:40.791607 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 732/1456] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 17 16:54:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:54:40.797703 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868a012-67c5-4c69-a80c-392c4da21deb None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:40.798253 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868a012-67c5-4c69-a80c-392c4da21deb None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:40.798253 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868a012-67c5-4c69-a80c-392c4da21deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:40.868404 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-a868a012-67c5-4c69-a80c-392c4da21deb None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:54:40.869680 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-a868a012-67c5-4c69-a80c-392c4da21deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:40.898643 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:40.967875 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 725/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 17 16:54:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:54:41.007111 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa796e5-0d94-4212-95ef-2b4d82b679c1 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:54:41.008660 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-dfa796e5-0d94-4212-95ef-2b4d82b679c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:41.115728 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa796e5-0d94-4212-95ef-2b4d82b679c1 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:54:41.144818 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:41.193131 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a52701-b08d-4cd7-a739-d28118fa1321 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9663edc8502b4aecb5317b24e4a5816b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:54:41.193798 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a52701-b08d-4cd7-a739-d28118fa1321 service neutron] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:41.194087 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a52701-b08d-4cd7-a739-d28118fa1321 service neutron] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:41.194366 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a52701-b08d-4cd7-a739-d28118fa1321 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:41.196042 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-e2a52701-b08d-4cd7-a739-d28118fa1321 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:41.307905 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a52701-b08d-4cd7-a739-d28118fa1321 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:54:41.309388 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a52701-b08d-4cd7-a739-d28118fa1321 service neutron] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:54:41.329168 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 733/1458] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:54:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:54:43.530445 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03886d8-e106-42a3-baac-276bc565cd56 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:43.531056 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03886d8-e106-42a3-baac-276bc565cd56 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:43.531056 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03886d8-e106-42a3-baac-276bc565cd56 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:43.532442 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 726/1459] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 17 16:54:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:54:43.539678 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29865500-241d-4577-afff-62dd0a8cd87e None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:43.540408 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29865500-241d-4577-afff-62dd0a8cd87e None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:43.540408 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29865500-241d-4577-afff-62dd0a8cd87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:43.617437 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-29865500-241d-4577-afff-62dd0a8cd87e None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:54:43.618732 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-29865500-241d-4577-afff-62dd0a8cd87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:43.628540 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:43.729667 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 734/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 17 16:54:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:54:43.768046 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03886d8-e106-42a3-baac-276bc565cd56 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:54:43.769652 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-c03886d8-e106-42a3-baac-276bc565cd56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:43.877440 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03886d8-e106-42a3-baac-276bc565cd56 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:54:43.935444 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:43.954434 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a2f93-0a99-44dc-8cac-c1dd35117216 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9663edc8502b4aecb5317b24e4a5816b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:54:43.955095 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a2f93-0a99-44dc-8cac-c1dd35117216 service neutron] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:43.955380 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a2f93-0a99-44dc-8cac-c1dd35117216 service neutron] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:43.955659 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a2f93-0a99-44dc-8cac-c1dd35117216 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:43.957349 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-e51a2f93-0a99-44dc-8cac-c1dd35117216 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:44.071887 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51a2f93-0a99-44dc-8cac-c1dd35117216 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:54:44.073381 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51a2f93-0a99-44dc-8cac-c1dd35117216 service neutron] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:54:44.091658 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 727/1461] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:54:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:54:46.188759 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f5d45-c672-45d2-9e63-8b0e385747f6 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:46.189311 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f5d45-c672-45d2-9e63-8b0e385747f6 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:46.189407 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f5d45-c672-45d2-9e63-8b0e385747f6 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:46.190782 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 735/1462] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 17 16:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:54:46.197079 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a302437-4376-4053-b81b-eb8b0953dc64 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:46.202639 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a302437-4376-4053-b81b-eb8b0953dc64 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:46.202639 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a302437-4376-4053-b81b-eb8b0953dc64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:46.273771 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-9a302437-4376-4053-b81b-eb8b0953dc64 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:54:46.275043 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-9a302437-4376-4053-b81b-eb8b0953dc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:46.320588 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:46.388070 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 728/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 17 16:54:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 17 16:54:46.427906 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f5d45-c672-45d2-9e63-8b0e385747f6 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:54:46.429532 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-541f5d45-c672-45d2-9e63-8b0e385747f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:46.536494 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f5d45-c672-45d2-9e63-8b0e385747f6 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:54:46.575199 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:46.613602 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae85f966-932f-455b-aa29-fb3baf8189bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9663edc8502b4aecb5317b24e4a5816b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:54:46.614735 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85f966-932f-455b-aa29-fb3baf8189bb service neutron] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:46.615328 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85f966-932f-455b-aa29-fb3baf8189bb service neutron] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:46.615653 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85f966-932f-455b-aa29-fb3baf8189bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:46.618049 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-ae85f966-932f-455b-aa29-fb3baf8189bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:46.732622 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae85f966-932f-455b-aa29-fb3baf8189bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:54:46.734094 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae85f966-932f-455b-aa29-fb3baf8189bb service neutron] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:54:46.753708 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 736/1464] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:54:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:54:48.944996 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5613f4-36e3-4b98-b2f2-d7cd40d83b3c None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:48.945576 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5613f4-36e3-4b98-b2f2-d7cd40d83b3c None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:48.945576 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5613f4-36e3-4b98-b2f2-d7cd40d83b3c None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:48.946958 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 729/1465] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 17 16:54:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:54:48.952446 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523437b-9de3-4fee-84b7-69d9fd6e1a83 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:48.952971 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523437b-9de3-4fee-84b7-69d9fd6e1a83 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:48.953080 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523437b-9de3-4fee-84b7-69d9fd6e1a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:49.022598 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-6523437b-9de3-4fee-84b7-69d9fd6e1a83 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:54:49.023833 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-6523437b-9de3-4fee-84b7-69d9fd6e1a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:49.042731 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:49.124359 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 737/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 17 16:54:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:54:49.161190 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5613f4-36e3-4b98-b2f2-d7cd40d83b3c None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:54:49.162988 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-5c5613f4-36e3-4b98-b2f2-d7cd40d83b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:49.275505 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5613f4-36e3-4b98-b2f2-d7cd40d83b3c None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:54:49.347166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:49.354975 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82dd269-6c58-43a0-8f13-d9d3d60131d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9663edc8502b4aecb5317b24e4a5816b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:54:49.361694 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82dd269-6c58-43a0-8f13-d9d3d60131d4 service neutron] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:49.361694 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82dd269-6c58-43a0-8f13-d9d3d60131d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:49.361694 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82dd269-6c58-43a0-8f13-d9d3d60131d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:49.361694 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-e82dd269-6c58-43a0-8f13-d9d3d60131d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:49.473980 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82dd269-6c58-43a0-8f13-d9d3d60131d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:54:49.475586 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82dd269-6c58-43a0-8f13-d9d3d60131d4 service neutron] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:54:49.494483 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 730/1467] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:54:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:54:51.555333 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b75fc-7698-404c-851e-cd4ee4ee9886 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:51.555902 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b75fc-7698-404c-851e-cd4ee4ee9886 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:51.555902 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b75fc-7698-404c-851e-cd4ee4ee9886 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:51.557322 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 738/1468] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 17 16:54:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:54:51.562774 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484ec64-dcf9-42e0-9f86-f61d72c1e8dc None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:51.563196 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484ec64-dcf9-42e0-9f86-f61d72c1e8dc None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:51.563339 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484ec64-dcf9-42e0-9f86-f61d72c1e8dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:51.631519 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-d484ec64-dcf9-42e0-9f86-f61d72c1e8dc None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:54:51.632753 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-d484ec64-dcf9-42e0-9f86-f61d72c1e8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:51.681108 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:51.732098 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 731/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 17 16:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:54:51.769587 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b75fc-7698-404c-851e-cd4ee4ee9886 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:54:51.771153 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-0f8b75fc-7698-404c-851e-cd4ee4ee9886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:51.891046 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b75fc-7698-404c-851e-cd4ee4ee9886 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:54:51.945803 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:54:51.974622 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b3b1a-9aba-4ba8-8d23-1da3b05e91db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9663edc8502b4aecb5317b24e4a5816b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:54:51.975283 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b3b1a-9aba-4ba8-8d23-1da3b05e91db service neutron] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:54:51.975569 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b3b1a-9aba-4ba8-8d23-1da3b05e91db service neutron] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:54:51.975845 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b3b1a-9aba-4ba8-8d23-1da3b05e91db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:54:51.977656 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-d13b3b1a-9aba-4ba8-8d23-1da3b05e91db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:54:52.096433 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b3b1a-9aba-4ba8-8d23-1da3b05e91db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:54:52.098051 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b3b1a-9aba-4ba8-8d23-1da3b05e91db service neutron] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:54:52.116596 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 739/1470] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:54:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:55:01.729969 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=22 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:55:02.114310 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=66 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:55:12.029333 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e288a0-ea44-4814-91cc-aa4bc71cdbd7 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:55:12.029917 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e288a0-ea44-4814-91cc-aa4bc71cdbd7 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:55:12.029917 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e288a0-ea44-4814-91cc-aa4bc71cdbd7 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:55:12.031286 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 732/1471] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:55:12.037561 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b8b58-e6f4-4621-9b96-30c90f4cc9f3 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:55:12.038028 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b8b58-e6f4-4621-9b96-30c90f4cc9f3 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:55:12.038124 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b8b58-e6f4-4621-9b96-30c90f4cc9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:55:12.106913 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-2c9b8b58-e6f4-4621-9b96-30c90f4cc9f3 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:55:12.108164 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-2c9b8b58-e6f4-4621-9b96-30c90f4cc9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:55:12.208316 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 740/1472] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:55:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:55:12.256749 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6ef5e-8017-4d83-a4fc-93a6ecba55cb None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:55:12.257067 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6ef5e-8017-4d83-a4fc-93a6ecba55cb None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:55:12.257351 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e6ef5e-8017-4d83-a4fc-93a6ecba55cb None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:55:12.258722 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 733/1473] 10.222.0.40 () {54 vars in 961 bytes} [Sun Jul 17 16:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:55:12.263634 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f9619-8db1-4d44-a964-f432d20e113e None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:55:12.263926 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f9619-8db1-4d44-a964-f432d20e113e None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:55:12.264206 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f9619-8db1-4d44-a964-f432d20e113e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:55:12.340321 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-f21f9619-8db1-4d44-a964-f432d20e113e None None] MFA Rules not processed for user `9e13bcc0f657445cbbdea5b78563f0d3`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:55:12.341475 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-f21f9619-8db1-4d44-a964-f432d20e113e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:55:12.363106 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:55:12.421277 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 741/1474] 10.222.0.40 () {58 vars in 1058 bytes} [Sun Jul 17 16:55:12 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:55:12.429238 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e6ef5e-8017-4d83-a4fc-93a6ecba55cb None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:55:12.430827 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-d9e6ef5e-8017-4d83-a4fc-93a6ecba55cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:55:12.539153 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e6ef5e-8017-4d83-a4fc-93a6ecba55cb None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:55:12.615548 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f273de4-a002-42cd-a654-13b1e2e7397b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e13bcc0f657445cbbdea5b78563f0d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:55:12.616123 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f273de4-a002-42cd-a654-13b1e2e7397b service glance] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:55:12.616406 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f273de4-a002-42cd-a654-13b1e2e7397b service glance] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:55:12.616683 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f273de4-a002-42cd-a654-13b1e2e7397b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:55:12.618398 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-0f273de4-a002-42cd-a654-13b1e2e7397b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:55:12.674964 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:55:12.732206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f273de4-a002-42cd-a654-13b1e2e7397b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:55:12.733689 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f273de4-a002-42cd-a654-13b1e2e7397b service glance] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:55:12.753818 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 734/1475] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:55:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:55:14.976811 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e8481-0171-4a4b-a664-b0a858c8c8a1 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:55:14.977405 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e8481-0171-4a4b-a664-b0a858c8c8a1 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:55:14.977405 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e8481-0171-4a4b-a664-b0a858c8c8a1 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:55:14.978783 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 742/1476] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:55:14.984481 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60198391-38b7-478b-b174-f41975b933e5 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:55:14.984959 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60198391-38b7-478b-b174-f41975b933e5 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:55:14.985057 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60198391-38b7-478b-b174-f41975b933e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:55:15.058081 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-60198391-38b7-478b-b174-f41975b933e5 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:55:15.059312 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-60198391-38b7-478b-b174-f41975b933e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:55:15.158286 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 735/1477] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:55:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:55:15.183086 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41e8481-0171-4a4b-a664-b0a858c8c8a1 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:55:15.184628 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-a41e8481-0171-4a4b-a664-b0a858c8c8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:55:15.317399 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41e8481-0171-4a4b-a664-b0a858c8c8a1 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:55:15.368153 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:55:15.416985 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfbd66b-8526-40b7-ad69-e48356c092ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b2ec585f5d6476389a82d18a223a808', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:55:15.417656 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbd66b-8526-40b7-ad69-e48356c092ec admin admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:55:15.417938 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbd66b-8526-40b7-ad69-e48356c092ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:55:15.418212 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbd66b-8526-40b7-ad69-e48356c092ec admin admin] PATH_INFO: `/v3/projects` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:55:15.418947 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfbd66b-8526-40b7-ad69-e48356c092ec admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:55:15.420735 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: WARNING py.warnings [None req-bdfbd66b-8526-40b7-ad69-e48356c092ec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 17 16:55:15.420735 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: warnings.warn(msg) Jul 17 16:55:15.420735 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: Jul 17 16:55:15.421888 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfbd66b-8526-40b7-ad69-e48356c092ec admin admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:55:15.443890 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 743/1478] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 17 16:55:15 2022] GET /identity/v3/projects => generated 2553 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 17 16:55:17.537816 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68efd9-f14c-4e54-bdba-03df56d39870 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:55:17.538376 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68efd9-f14c-4e54-bdba-03df56d39870 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:55:17.538376 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68efd9-f14c-4e54-bdba-03df56d39870 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:55:17.539742 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 736/1479] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:55:17.545156 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b49e5-1847-4f17-8231-fd31aae33324 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:55:17.545641 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b49e5-1847-4f17-8231-fd31aae33324 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:55:17.545763 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b49e5-1847-4f17-8231-fd31aae33324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:55:17.613220 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-1f7b49e5-1847-4f17-8231-fd31aae33324 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:55:17.614439 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-1f7b49e5-1847-4f17-8231-fd31aae33324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:55:17.711091 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 744/1480] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:55:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:55:17.754103 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68efd9-f14c-4e54-bdba-03df56d39870 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:55:17.755661 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-cc68efd9-f14c-4e54-bdba-03df56d39870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:55:17.765591 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:55:17.905307 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68efd9-f14c-4e54-bdba-03df56d39870 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:55:17.989902 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:55:18.032770 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722eacbf-a9de-4202-9784-2a1c9c34627e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03244087f1a34bddb11ea46aba11a148', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:55:18.033499 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722eacbf-a9de-4202-9784-2a1c9c34627e service nova] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:55:18.034315 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722eacbf-a9de-4202-9784-2a1c9c34627e service nova] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:55:18.034603 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722eacbf-a9de-4202-9784-2a1c9c34627e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:55:18.036499 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-722eacbf-a9de-4202-9784-2a1c9c34627e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:55:18.129389 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722eacbf-a9de-4202-9784-2a1c9c34627e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:55:18.130995 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722eacbf-a9de-4202-9784-2a1c9c34627e service nova] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:55:18.149791 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 737/1481] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:55:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:55:20.397910 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e0689-3748-43ee-8a6e-9f55f32c4bbd None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:55:20.398423 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e0689-3748-43ee-8a6e-9f55f32c4bbd None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:55:20.398423 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e0689-3748-43ee-8a6e-9f55f32c4bbd None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:55:20.399770 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 745/1482] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:55:20.405210 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509f561-63f3-491d-88af-d5177f04c387 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:55:20.405727 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509f561-63f3-491d-88af-d5177f04c387 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:55:20.405831 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509f561-63f3-491d-88af-d5177f04c387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:55:20.476074 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-0509f561-63f3-491d-88af-d5177f04c387 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:55:20.477418 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-0509f561-63f3-491d-88af-d5177f04c387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:55:20.527680 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:55:20.582478 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 738/1483] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:55:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:55:20.619435 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7e0689-3748-43ee-8a6e-9f55f32c4bbd None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:55:20.621026 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-8a7e0689-3748-43ee-8a6e-9f55f32c4bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:55:20.645648 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:55:20.732867 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7e0689-3748-43ee-8a6e-9f55f32c4bbd None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:55:20.811714 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb60d81f-9b8a-471a-95aa-74df679c3ddf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9663edc8502b4aecb5317b24e4a5816b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:55:20.812384 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60d81f-9b8a-471a-95aa-74df679c3ddf service neutron] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:55:20.812667 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60d81f-9b8a-471a-95aa-74df679c3ddf service neutron] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:55:20.812983 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb60d81f-9b8a-471a-95aa-74df679c3ddf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:55:20.814696 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-cb60d81f-9b8a-471a-95aa-74df679c3ddf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:55:20.893293 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:55:20.929969 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb60d81f-9b8a-471a-95aa-74df679c3ddf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:55:20.931417 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb60d81f-9b8a-471a-95aa-74df679c3ddf service neutron] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:55:20.951187 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 746/1484] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:55:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:55:23.062364 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b81bf-2425-45b3-bbf9-ece41652f1c2 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:55:23.062892 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b81bf-2425-45b3-bbf9-ece41652f1c2 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:55:23.062892 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b81bf-2425-45b3-bbf9-ece41652f1c2 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:55:23.064321 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 739/1485] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:55:23.070411 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246154e2-933c-4537-9b31-8f3218a5fe88 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:55:23.070866 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246154e2-933c-4537-9b31-8f3218a5fe88 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:55:23.070984 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246154e2-933c-4537-9b31-8f3218a5fe88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:55:23.143974 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-246154e2-933c-4537-9b31-8f3218a5fe88 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:55:23.145474 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-246154e2-933c-4537-9b31-8f3218a5fe88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:55:23.248196 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 747/1486] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:55:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:55:23.287597 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b81bf-2425-45b3-bbf9-ece41652f1c2 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:55:23.289213 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-c39b81bf-2425-45b3-bbf9-ece41652f1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:55:23.396548 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b81bf-2425-45b3-bbf9-ece41652f1c2 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:55:23.474512 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:55:23.476720 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554aa751-d0a3-4b79-9361-409efecc6813 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9663edc8502b4aecb5317b24e4a5816b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:55:23.477415 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554aa751-d0a3-4b79-9361-409efecc6813 service neutron] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:55:23.477730 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554aa751-d0a3-4b79-9361-409efecc6813 service neutron] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:55:23.478010 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554aa751-d0a3-4b79-9361-409efecc6813 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:55:23.479771 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-554aa751-d0a3-4b79-9361-409efecc6813 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:55:23.602939 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554aa751-d0a3-4b79-9361-409efecc6813 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:55:23.604390 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554aa751-d0a3-4b79-9361-409efecc6813 service neutron] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:55:23.622823 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 740/1487] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:55:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:55:25.776774 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de725826-9fa5-4521-afa4-525709ecbe37 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:55:25.777367 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de725826-9fa5-4521-afa4-525709ecbe37 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:55:25.777367 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de725826-9fa5-4521-afa4-525709ecbe37 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:55:25.778806 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 748/1488] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:55:25.784810 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8793e-bb5b-49b4-8c2d-8c020fcab375 None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:55:25.785324 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8793e-bb5b-49b4-8c2d-8c020fcab375 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:55:25.785453 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8793e-bb5b-49b4-8c2d-8c020fcab375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:55:25.853123 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-44a8793e-bb5b-49b4-8c2d-8c020fcab375 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:55:25.854364 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-44a8793e-bb5b-49b4-8c2d-8c020fcab375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:55:25.913425 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:55:25.961301 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 741/1489] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:55:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:55:26.001831 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de725826-9fa5-4521-afa4-525709ecbe37 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:55:26.004899 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-de725826-9fa5-4521-afa4-525709ecbe37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:55:26.047498 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:55:26.119981 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de725826-9fa5-4521-afa4-525709ecbe37 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:55:26.200391 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64072cb5-e14b-43b7-9e59-75f258802d14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9663edc8502b4aecb5317b24e4a5816b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:55:26.201085 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64072cb5-e14b-43b7-9e59-75f258802d14 service neutron] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:55:26.201373 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64072cb5-e14b-43b7-9e59-75f258802d14 service neutron] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:55:26.201653 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64072cb5-e14b-43b7-9e59-75f258802d14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:55:26.203319 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-64072cb5-e14b-43b7-9e59-75f258802d14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:55:26.296384 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:55:26.327728 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64072cb5-e14b-43b7-9e59-75f258802d14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:55:26.329151 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64072cb5-e14b-43b7-9e59-75f258802d14 service neutron] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:55:26.351497 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 749/1490] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:55:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:55:28.756980 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6da37-be3d-4efc-a399-fbb6896991ae None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:55:28.757606 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6da37-be3d-4efc-a399-fbb6896991ae None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:55:28.757606 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6da37-be3d-4efc-a399-fbb6896991ae None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:55:28.758914 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 742/1491] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 17 16:55:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:55:28.769275 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef08697-cd22-4a8a-a836-9a6ef16737ff None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:55:28.769781 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef08697-cd22-4a8a-a836-9a6ef16737ff None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:55:28.769907 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef08697-cd22-4a8a-a836-9a6ef16737ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:55:28.835043 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-bef08697-cd22-4a8a-a836-9a6ef16737ff None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:55:28.836269 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-bef08697-cd22-4a8a-a836-9a6ef16737ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:55:28.935320 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 750/1492] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 17 16:55:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 17 16:55:28.975427 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c6da37-be3d-4efc-a399-fbb6896991ae None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:55:28.977053 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-51c6da37-be3d-4efc-a399-fbb6896991ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:55:29.083700 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c6da37-be3d-4efc-a399-fbb6896991ae None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:55:29.156456 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb90fa-9911-4372-b2e8-aa6657c6360f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9663edc8502b4aecb5317b24e4a5816b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:55:29.157140 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb90fa-9911-4372-b2e8-aa6657c6360f service neutron] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:55:29.157428 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb90fa-9911-4372-b2e8-aa6657c6360f service neutron] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:55:29.157894 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb90fa-9911-4372-b2e8-aa6657c6360f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:55:29.160120 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-5adb90fa-9911-4372-b2e8-aa6657c6360f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:55:29.169587 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=100 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:55:29.275987 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb90fa-9911-4372-b2e8-aa6657c6360f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:55:29.277531 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb90fa-9911-4372-b2e8-aa6657c6360f service neutron] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:55:29.300202 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 743/1493] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:55:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:55:38.933551 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=82 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:55:39.296739 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=50 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:57:35.740293 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa07df7-d2af-49ef-9c89-9ea63fb071e3 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:57:35.740937 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa07df7-d2af-49ef-9c89-9ea63fb071e3 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:57:35.740937 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa07df7-d2af-49ef-9c89-9ea63fb071e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:57:35.821804 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-8aa07df7-d2af-49ef-9c89-9ea63fb071e3 None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:57:35.823025 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-8aa07df7-d2af-49ef-9c89-9ea63fb071e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:57:35.928434 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 751/1494] 10.222.0.40 () {58 vars in 957 bytes} [Sun Jul 17 16:57:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:57:35.945339 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868606b5-cae5-4dbd-b458-69ec6e959dda None None] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:57:35.945899 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868606b5-cae5-4dbd-b458-69ec6e959dda None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:57:35.945899 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868606b5-cae5-4dbd-b458-69ec6e959dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:57:35.967473 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.auth.core [None req-868606b5-cae5-4dbd-b458-69ec6e959dda None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:57:35.967473 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-868606b5-cae5-4dbd-b458-69ec6e959dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:57:35.975885 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 744/1495] 10.222.0.40 () {58 vars in 957 bytes} [Sun Jul 17 16:57:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:57:35.981400 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa07df7-d2af-49ef-9c89-9ea63fb071e3 None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:57:35.991218 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa07df7-d2af-49ef-9c89-9ea63fb071e3 None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:57:35.996596 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65ccd9-701a-4e96-a14e-2b81efa1b1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b2ec585f5d6476389a82d18a223a808', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:57:35.997302 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65ccd9-701a-4e96-a14e-2b81efa1b1df admin admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:57:35.997596 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65ccd9-701a-4e96-a14e-2b81efa1b1df admin admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:57:35.997881 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65ccd9-701a-4e96-a14e-2b81efa1b1df admin admin] PATH_INFO: `/v3/domains` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:57:35.998626 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65ccd9-701a-4e96-a14e-2b81efa1b1df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87198) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 17 16:57:35.999025 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65ccd9-701a-4e96-a14e-2b81efa1b1df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:57:36.000082 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: WARNING py.warnings [None req-cc65ccd9-701a-4e96-a14e-2b81efa1b1df admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 17 16:57:36.000082 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: warnings.warn(msg) Jul 17 16:57:36.000082 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: Jul 17 16:57:36.001014 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65ccd9-701a-4e96-a14e-2b81efa1b1df admin admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:57:36.017538 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 752/1496] 10.222.0.40 () {58 vars in 1152 bytes} [Sun Jul 17 16:57:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 17 16:57:36.025867 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868606b5-cae5-4dbd-b458-69ec6e959dda None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:57:36.035427 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868606b5-cae5-4dbd-b458-69ec6e959dda None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:57:36.040166 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79478e34-3f4f-4baf-9da4-c2c9f00f29b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b2ec585f5d6476389a82d18a223a808', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:57:36.051856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79478e34-3f4f-4baf-9da4-c2c9f00f29b9 admin admin] REQUEST_METHOD: `POST` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:57:36.051856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79478e34-3f4f-4baf-9da4-c2c9f00f29b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:57:36.051856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79478e34-3f4f-4baf-9da4-c2c9f00f29b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:57:36.051856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79478e34-3f4f-4baf-9da4-c2c9f00f29b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:57:36.051856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: WARNING py.warnings [None req-79478e34-3f4f-4baf-9da4-c2c9f00f29b9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 17 16:57:36.051856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: warnings.warn(msg) Jul 17 16:57:36.051856 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: Jul 17 16:57:36.052445 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79478e34-3f4f-4baf-9da4-c2c9f00f29b9 admin admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:57:36.128720 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 745/1497] 10.222.0.40 () {60 vars in 1152 bytes} [Sun Jul 17 16:57:36 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:57:36.132402 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65ccd9-701a-4e96-a14e-2b81efa1b1df admin admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:57:36.142987 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65ccd9-701a-4e96-a14e-2b81efa1b1df admin admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:57:36.195816 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=100 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:57:36.202855 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f91f7a-52d7-4a0d-945d-6d3414415eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b2ec585f5d6476389a82d18a223a808', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:57:36.203537 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f91f7a-52d7-4a0d-945d-6d3414415eae admin admin] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:57:36.203827 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f91f7a-52d7-4a0d-945d-6d3414415eae admin admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:57:36.204293 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f91f7a-52d7-4a0d-945d-6d3414415eae admin admin] PATH_INFO: `/v3/users` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:57:36.205233 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f91f7a-52d7-4a0d-945d-6d3414415eae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:57:36.206344 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: WARNING py.warnings [None req-12f91f7a-52d7-4a0d-945d-6d3414415eae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 17 16:57:36.206344 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: warnings.warn(msg) Jul 17 16:57:36.206344 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: Jul 17 16:57:36.207514 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f91f7a-52d7-4a0d-945d-6d3414415eae admin admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:57:36.326844 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 753/1498] 10.222.0.40 () {60 vars in 1143 bytes} [Sun Jul 17 16:57:36 2022] POST /identity/v3/users => generated 289 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 17 16:57:36.331486 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79478e34-3f4f-4baf-9da4-c2c9f00f29b9 admin admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:57:36.341950 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79478e34-3f4f-4baf-9da4-c2c9f00f29b9 admin admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:57:36.346545 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f7486-a706-45c0-8fb9-5e4dd9aa86f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b2ec585f5d6476389a82d18a223a808', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:57:36.347206 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f7486-a706-45c0-8fb9-5e4dd9aa86f4 admin admin] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:57:36.347497 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f7486-a706-45c0-8fb9-5e4dd9aa86f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:57:36.347770 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f7486-a706-45c0-8fb9-5e4dd9aa86f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:57:36.348504 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f7486-a706-45c0-8fb9-5e4dd9aa86f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:57:36.349567 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: WARNING py.warnings [None req-ba3f7486-a706-45c0-8fb9-5e4dd9aa86f4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 17 16:57:36.349567 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: warnings.warn(msg) Jul 17 16:57:36.349567 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: Jul 17 16:57:36.356619 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f7486-a706-45c0-8fb9-5e4dd9aa86f4 admin admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:57:36.363738 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 746/1499] 10.222.0.40 () {58 vars in 1121 bytes} [Sun Jul 17 16:57:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 17 16:57:36.367762 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f91f7a-52d7-4a0d-945d-6d3414415eae admin admin] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:57:36.378574 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f91f7a-52d7-4a0d-945d-6d3414415eae admin admin] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:57:36.383193 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d029c79-a6ac-40cb-b1e5-12bfcdf7996d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b2ec585f5d6476389a82d18a223a808', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:57:36.383852 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d029c79-a6ac-40cb-b1e5-12bfcdf7996d admin admin] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:57:36.384141 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d029c79-a6ac-40cb-b1e5-12bfcdf7996d admin admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:57:36.384418 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d029c79-a6ac-40cb-b1e5-12bfcdf7996d admin admin] PATH_INFO: `/v3/roles` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:57:36.385192 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d029c79-a6ac-40cb-b1e5-12bfcdf7996d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:57:36.386257 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: WARNING py.warnings [None req-7d029c79-a6ac-40cb-b1e5-12bfcdf7996d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 17 16:57:36.386257 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: warnings.warn(msg) Jul 17 16:57:36.386257 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: Jul 17 16:57:36.387132 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d029c79-a6ac-40cb-b1e5-12bfcdf7996d admin admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:57:36.400589 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 754/1500] 10.222.0.40 () {58 vars in 1121 bytes} [Sun Jul 17 16:57:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 17 16:57:36.404797 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f7486-a706-45c0-8fb9-5e4dd9aa86f4 admin admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:57:36.415020 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f7486-a706-45c0-8fb9-5e4dd9aa86f4 admin admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:57:36.420447 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a35dfc-2b21-4f45-82ee-e89c68ecd51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b2ec585f5d6476389a82d18a223a808', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:57:36.421361 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a35dfc-2b21-4f45-82ee-e89c68ecd51c admin admin] REQUEST_METHOD: `PUT` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:57:36.421648 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a35dfc-2b21-4f45-82ee-e89c68ecd51c admin admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:57:36.421929 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a35dfc-2b21-4f45-82ee-e89c68ecd51c admin admin] PATH_INFO: `/v3/projects/87110b80694448a0ab174af6530661e0/users/79b84fdbf4524dd985dfd1f3cbb3eb07/roles/a47f5df8505f49419d0acb4f0f287610` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:57:36.437830 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a35dfc-2b21-4f45-82ee-e89c68ecd51c admin admin] RBAC: Authorizing `identity:create_grant(project_id=87110b80694448a0ab174af6530661e0, user_id=79b84fdbf4524dd985dfd1f3cbb3eb07, role_id=a47f5df8505f49419d0acb4f0f287610)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:57:36.438853 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: WARNING py.warnings [None req-d8a35dfc-2b21-4f45-82ee-e89c68ecd51c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 17 16:57:36.438853 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: warnings.warn(msg) Jul 17 16:57:36.438853 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: Jul 17 16:57:36.440775 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a35dfc-2b21-4f45-82ee-e89c68ecd51c admin admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:57:36.520735 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 747/1501] 10.222.0.40 () {60 vars in 1482 bytes} [Sun Jul 17 16:57:36 2022] PUT /identity/v3/projects/87110b80694448a0ab174af6530661e0/users/79b84fdbf4524dd985dfd1f3cbb3eb07/roles/a47f5df8505f49419d0acb4f0f287610 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 17 16:57:36.533516 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384ad31-b68d-4a6f-80f8-4282d98e0766 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:57:36.533934 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384ad31-b68d-4a6f-80f8-4282d98e0766 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:57:36.534265 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384ad31-b68d-4a6f-80f8-4282d98e0766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:57:36.564422 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-3384ad31-b68d-4a6f-80f8-4282d98e0766 None None] MFA Rules not processed for user `79b84fdbf4524dd985dfd1f3cbb3eb07`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:57:36.565690 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-3384ad31-b68d-4a6f-80f8-4282d98e0766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:57:36.631031 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 755/1502] 10.222.0.40 () {58 vars in 957 bytes} [Sun Jul 17 16:57:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:57:36.645479 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd3a38-76cb-49c4-b618-f2521e4cf350 None None] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:57:36.645784 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd3a38-76cb-49c4-b618-f2521e4cf350 None None] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:57:36.646067 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd3a38-76cb-49c4-b618-f2521e4cf350 None None] PATH_INFO: `/` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:57:36.647450 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 748/1503] 10.222.0.40 () {54 vars in 964 bytes} [Sun Jul 17 16:57:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:57:36.652699 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07399b6-f5a7-4355-bce2-c3164eca0e48 None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:57:36.653008 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07399b6-f5a7-4355-bce2-c3164eca0e48 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:57:36.653282 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07399b6-f5a7-4355-bce2-c3164eca0e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:57:36.716065 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-a07399b6-f5a7-4355-bce2-c3164eca0e48 None None] MFA Rules not processed for user `0b7122fc744d446aa60b25844c68f15b`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:57:36.717244 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-a07399b6-f5a7-4355-bce2-c3164eca0e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:57:36.796148 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 756/1504] 10.222.0.40 () {58 vars in 1061 bytes} [Sun Jul 17 16:57:36 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:57:36.804050 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd3a38-76cb-49c4-b618-f2521e4cf350 None None] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:57:36.815289 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd3a38-76cb-49c4-b618-f2521e4cf350 None None] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:57:36.818914 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c135699-4ca0-4bd8-8a1d-8d7f2f97b487 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b7122fc744d446aa60b25844c68f15b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:57:36.819561 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c135699-4ca0-4bd8-8a1d-8d7f2f97b487 service cinder] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:57:36.819849 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c135699-4ca0-4bd8-8a1d-8d7f2f97b487 service cinder] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:57:36.820139 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c135699-4ca0-4bd8-8a1d-8d7f2f97b487 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:57:36.829698 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c135699-4ca0-4bd8-8a1d-8d7f2f97b487 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:57:36.831168 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c135699-4ca0-4bd8-8a1d-8d7f2f97b487 service cinder] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:57:36.840421 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 749/1505] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:57:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:57:36.884677 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2302f-0203-42f7-87d6-4d9fa2dd81b8 None None] REQUEST_METHOD: `GET` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:57:36.884998 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2302f-0203-42f7-87d6-4d9fa2dd81b8 None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:57:36.885283 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2302f-0203-42f7-87d6-4d9fa2dd81b8 None None] PATH_INFO: `/` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:57:36.886718 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 757/1506] 10.222.0.40 () {52 vars in 829 bytes} [Sun Jul 17 16:57:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 17 16:57:36.890337 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c135699-4ca0-4bd8-8a1d-8d7f2f97b487 service cinder] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:57:36.954740 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c135699-4ca0-4bd8-8a1d-8d7f2f97b487 service cinder] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:57:36.959419 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7823c54f-3baf-45af-8e78-06e58b7f4c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b2ec585f5d6476389a82d18a223a808', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:57:36.960289 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7823c54f-3baf-45af-8e78-06e58b7f4c11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:57:36.960569 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7823c54f-3baf-45af-8e78-06e58b7f4c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:57:36.960950 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7823c54f-3baf-45af-8e78-06e58b7f4c11 admin admin] PATH_INFO: `/v3/users/79b84fdbf4524dd985dfd1f3cbb3eb07` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:57:36.962099 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7823c54f-3baf-45af-8e78-06e58b7f4c11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79b84fdbf4524dd985dfd1f3cbb3eb07)` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:57:36.963117 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: WARNING py.warnings [None req-7823c54f-3baf-45af-8e78-06e58b7f4c11 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 17 16:57:36.963117 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: warnings.warn(msg) Jul 17 16:57:36.963117 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: Jul 17 16:57:36.964876 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7823c54f-3baf-45af-8e78-06e58b7f4c11 admin admin] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:57:37.206582 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [None req-7823c54f-3baf-45af-8e78-06e58b7f4c11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79b84fdbf4524dd985dfd1f3cbb3eb07'} {{(pid=87199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 17 16:57:37.223014 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [None req-7823c54f-3baf-45af-8e78-06e58b7f4c11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79b84fdbf4524dd985dfd1f3cbb3eb07'} {{(pid=87199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 17 16:57:37.223862 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [None req-7823c54f-3baf-45af-8e78-06e58b7f4c11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79b84fdbf4524dd985dfd1f3cbb3eb07'} {{(pid=87199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 17 16:57:37.224500 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [None req-7823c54f-3baf-45af-8e78-06e58b7f4c11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79b84fdbf4524dd985dfd1f3cbb3eb07'} {{(pid=87199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 17 16:57:37.246601 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.notifications [None req-7823c54f-3baf-45af-8e78-06e58b7f4c11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79b84fdbf4524dd985dfd1f3cbb3eb07'} {{(pid=87199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 17 16:57:37.250974 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:57:37.347770 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 750/1507] 10.222.0.40 () {58 vars in 1223 bytes} [Sun Jul 17 16:57:36 2022] DELETE /identity/v3/users/79b84fdbf4524dd985dfd1f3cbb3eb07 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 17 16:57:37.352329 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8f102-6b21-4dfb-8928-fb8a3d02939f None None] REQUEST_METHOD: `POST` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:57:37.352614 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8f102-6b21-4dfb-8928-fb8a3d02939f None None] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:57:37.352936 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8f102-6b21-4dfb-8928-fb8a3d02939f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:57:37.372236 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.auth.core [None req-3ea8f102-6b21-4dfb-8928-fb8a3d02939f None None] MFA Rules not processed for user `45e7917424634d8897c1eb02702d4bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87198) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 17 16:57:37.373515 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-3ea8f102-6b21-4dfb-8928-fb8a3d02939f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:57:37.383707 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:57:37.436472 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 758/1508] 10.222.0.40 () {58 vars in 957 bytes} [Sun Jul 17 16:57:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 17 16:57:37.446134 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7823c54f-3baf-45af-8e78-06e58b7f4c11 admin admin] Authenticating user token {{(pid=87199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:57:37.447644 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.fernet_utils [None req-7823c54f-3baf-45af-8e78-06e58b7f4c11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:57:37.534046 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7823c54f-3baf-45af-8e78-06e58b7f4c11 admin admin] Validating token access rules against request {{(pid=87199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:57:37.537779 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f139531-e180-4ce3-8eb4-d309b7580e1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9663edc8502b4aecb5317b24e4a5816b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '927057f9a5af4833aeac9992203a223c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:57:37.538441 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f139531-e180-4ce3-8eb4-d309b7580e1f service neutron] REQUEST_METHOD: `GET` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:57:37.538731 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f139531-e180-4ce3-8eb4-d309b7580e1f service neutron] SCRIPT_NAME: `/identity` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:57:37.539018 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f139531-e180-4ce3-8eb4-d309b7580e1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:57:37.549414 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f139531-e180-4ce3-8eb4-d309b7580e1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:57:37.550875 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f139531-e180-4ce3-8eb4-d309b7580e1f service neutron] RBAC: Authorization granted {{(pid=87199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:57:37.560741 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: [pid: 87199|app: 0|req: 751/1509] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 17 16:57:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 17 16:57:38.068497 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea8f102-6b21-4dfb-8928-fb8a3d02939f None None] Authenticating user token {{(pid=87198) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 17 16:57:38.069997 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.fernet_utils [None req-3ea8f102-6b21-4dfb-8928-fb8a3d02939f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87198) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 17 16:57:38.082606 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea8f102-6b21-4dfb-8928-fb8a3d02939f None None] Validating token access rules against request {{(pid=87198) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 17 16:57:38.088127 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13d7fb-cf57-42dd-8868-8017f82001ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e7917424634d8897c1eb02702d4bc2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3b2ec585f5d6476389a82d18a223a808', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87198) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 17 16:57:38.089488 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13d7fb-cf57-42dd-8868-8017f82001ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 17 16:57:38.089845 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13d7fb-cf57-42dd-8868-8017f82001ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 17 16:57:38.090193 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13d7fb-cf57-42dd-8868-8017f82001ac admin admin] PATH_INFO: `/v3/projects/87110b80694448a0ab174af6530661e0` {{(pid=87198) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 17 16:57:38.094384 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be13d7fb-cf57-42dd-8868-8017f82001ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=87110b80694448a0ab174af6530661e0)` {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 17 16:57:38.094384 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: WARNING py.warnings [None req-be13d7fb-cf57-42dd-8868-8017f82001ac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 17 16:57:38.094384 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: warnings.warn(msg) Jul 17 16:57:38.094384 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: Jul 17 16:57:38.095411 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be13d7fb-cf57-42dd-8868-8017f82001ac admin admin] RBAC: Authorization granted {{(pid=87198) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 17 16:57:38.229513 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [None req-be13d7fb-cf57-42dd-8868-8017f82001ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87110b80694448a0ab174af6530661e0', 'request_id': 'req-be13d7fb-cf57-42dd-8868-8017f82001ac'} {{(pid=87198) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 17 16:57:38.235586 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [None req-be13d7fb-cf57-42dd-8868-8017f82001ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87110b80694448a0ab174af6530661e0', 'request_id': 'req-be13d7fb-cf57-42dd-8868-8017f82001ac'} {{(pid=87198) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 17 16:57:38.251032 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [None req-be13d7fb-cf57-42dd-8868-8017f82001ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87110b80694448a0ab174af6530661e0', 'request_id': 'req-be13d7fb-cf57-42dd-8868-8017f82001ac'} {{(pid=87198) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 17 16:57:38.344159 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [None req-be13d7fb-cf57-42dd-8868-8017f82001ac admin admin] The token cache is being invalidate because project 87110b80694448a0ab174af6530661e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87198) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 17 16:57:38.344409 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [None req-be13d7fb-cf57-42dd-8868-8017f82001ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87198) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 17 16:57:38.345115 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG keystone.notifications [None req-be13d7fb-cf57-42dd-8868-8017f82001ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87198) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 17 16:57:38.346869 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: [pid: 87198|app: 0|req: 759/1510] 10.222.0.40 () {58 vars in 1232 bytes} [Sun Jul 17 16:57:38 2022] DELETE /identity/v3/projects/87110b80694448a0ab174af6530661e0 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 17 16:57:47.556169 dv-cinder-850015-2-iscsi devstack@keystone.service[87199]: DEBUG dbcounter [-] [87199] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=87199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 17 16:57:48.340719 dv-cinder-850015-2-iscsi devstack@keystone.service[87198]: DEBUG dbcounter [-] [87198] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87198) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}